Check+manipulate plugin config at mount time

[markus2330]

I propose to add a functionality that plugins can validate or even manipulate their configuration at mount time. We already use that for the resolver (checkfile) and it turns out that such feature is essential for the crypto plugin which needs to encrypt a password at mount time.

We could introduce following convention for plugins: Whenever a plugin exports the symbol checkconf (using exports/checkconf), this function with the signature int checkconf (Key * errorKey, KeySet * config) will be called.

Alternative: Add a pseudo-key /checkconf to kdbOpen which will then switch kdbOpen to a configuration validator and does not actually open the plugin. This alternative, however, might overload kdbOpen too much. (It already has the /modules feature to open a plugin even though it has no config.)

Within checkconf plugins can validate their proposed configuration and reject it if unfeasible. E.g. when a user passes a ID for a public key, checkconf tries to encrypt with it. If it does not work, it would return -1 and set an error (and propose valid IDs within the error message). Then the user can retry with a different ID. Furthermore, checkconf can manipulate its config, i.e. it can add an encrypted random-generated value as needed for the crypto plugin.

@petermax2: what do you think?

[petermax2]

We could introduce following convention for plugins: Whenever a plugin exports the symbol checkconf (using exports/checkconf), this function with the signature int checkconf (Key * errorKey, KeySet * config) will be called.

This would be very helpful for the crypto plugin. 👍

[markus2330]

unfortunately not this release, @petermax2 do you have resources to do it? Let us talk about it tomorrow.

[markus2330]

checkconf would also make checkfile of the resolvers unnecessary (its only a special case of checkconf.

[petermax2]

I understand the concept of checkfile but I am not sure where to put checkconf.

Is this a good location for calling checkconf?

// src/libs/tools/src/backendbuilder.cpp:391
void BackendBuilder::addPlugin (PluginSpec const & plugin)

While adding each plugin to the backend, every plugin exporting the exports/checkconf symbol could verify its own configuration or call the gpg-binary whatsoever.

Or would you rather introduce a new method within the BackendBuilder that iterates over all plugins and calls checkconf where applicable? I would then call this method in src/tools/kdb/mount.cpp:140 within the method MountCommand::buildBackend( ) right after

backend.addPlugins (parseArguments (cl.plugins));

[markus2330]

Yeah, I know its a bit tricky because the BackendBuilders are quite nested and helpful documentation is missing. Actually, both of your suggestions are sound and useful. From usability point of view addPlugin might be even the better place:

1. ABI stays the same
2. API usage is nearly the same, with possibly one minor exception: If we want to consider backendConfig, then it would be preferable to use setBackendConfig before addPlugins, so that it can be considered properly (different history constraint). It is currently not documented, kdb mount does it this way, but kdb mount-spec not. We could also say that only the plugin config is validated, which would force the plugin config to be complete (without having the backend config as fallback).
3. For interactive adding of plugins, the error would appear exactly when the user wants to add the plugin, with immediate feedback when it does not validate. It might even work in qt-gui without modifications.

[petermax2]

Is there a way to retrieve the exported symbols (provided by the plugin) from kdb::tools::PluginSpec?

At the place where checkfile is called we are dealing with kdb::tools::PluginPtr. In the backend builder (specifically within the method addPlugins) we have PluginSpecs.

[markus2330]

No, you have to create a plugin. Please abstract the getSymbol in the pluginDatabase. Both status and lookupInfo already create plugins, it is a one-liner. The pluginDatabase abstraction is useful for both unit tests and to make it easier in future to avoid the many plugin creations. (If you feel like it, you can make that optimization, too.)

[petermax2]

Alright, I'll try. Thank you for the quick response!

[petermax2]

I added an example checkconf to the "template" plugin (see 9542f02). Which parts of the documentation should mention checkconf? I will update them in the evening.

[markus2330]

Thank you!

Can you write "this function is optional" to other optional functions, too? (open, close, error, set) I think "if the export symbol "exports/checkconf" is provided" is not needed there, because the symbol is already provided in the template. Its easer to give feedback on a PR, though.

Following parts should mention it:

• doc/tutorials/plugins.md
• plugin doc (I think kdbOpen should also use checkconf it to validate its own config)