Review configuration mechanism

[sbellem]

The current configuration mechanism has some strength and weaknesses. One of its current strengths appears to be that it is very convenient as it happens automatically in the root __init__.py:

https://github.com/initc3/HoneyBadgerMPC/blob/0914c0269a5d3636b01f40557e8ea432f721bd04/honeybadgermpc/__init__.py#L23

This comes at a cost though: building documentation that has docstrings fails (see #378). Perhaps it could also be a cause to other problems in the future but for now that seems to be the only problem. So one may object that it is a very low cost in exchange for the convenience it provides. Perhaps. Nevertheless, conceptually speaking it seems that we could improve the current mechanism in place to not only restore the capability of building documentation but to also make the configuration mechanism clearer, more explicit, and re-usable as needed in the modules that require it.

Rough Planning Notes

• [x] Review and understand how the current mechanism works.
• [x] Identify what commands/entrypoints require loading the configuration from a file.
• [x] Clarify whether there is one or multiple entrypoints to the system. Does one always starts the system via the same entrypoint or is the current code base more like a framework/toolbox with multiple independent component each having its one "entry point" but nevertheless sharing the same configuration object/mechanism.
• [x] Implement a different mechanism so that #378 can be resolved. - [ ] Address #222.

Observations

The main observation one may make is that the "loading config" code that reads command line arguments does not need to be invoked by default in the project root __init__.py.

Questions

• Would it make sense to transfer the responsibility of load_config to a more general command line argument?

• Would it make sense to have one top command, say hbmpc that can take in subcommands? For example: hbmpc run appname ....

• If the current code base has multiple standalone "apps" each with its own command that depends on the current HbmpcConfig then would it make sense to have an overall general command line interface that includes the option of loading a config as it is currently done, and each "app" that requires its own command line could inherit from the general command line interface with the already existing configuration mechanism "baked in". This approach would achieve a very similar convenience as the current approach but would limit the loading of the config to actual cases where it is indeed needed. For non-command line cases a default config may do the work.

Configuration Mechanisms in other Projects

How do other projects do it? Which projects are similar in terms of structure and patterns so that it makes to look at how they handle the configuration mechanism?

• docker-compose -- not sure, but looking at it may provide some perspective.
• saltstack -- It is complicated but may be useful to look at still.
• pytest -- Seems counter-intuitive to look at pytest but maybe that is actually a very decent place to look for inspiration.

[sbellem]

Review and understand how the current mechanism works

Applications making use of HbmpcConfig:

• apps/asynchromix/butterfly_network.py
• apps/asynchromix/powermixing.py
• apps/tutorial/hbmpc-tutorial-2.py
• honeybadgermpc/ipc.py
• honeybadgermpc/offline_randousha.py
• honeybadgermpc/broadcast/binaryagreement.py
• honeybadgermpc/broadcast/commoncoin.py
• honeybadgermpc/broadcast/commonsubset.py
• honeybadgermpc/offline_robust.py
• scripts/hbavss_batch.py
• scripts/hbavss_light.py

It seems that the modules that depend on HbmpcConfig are meant to invoked by a higher-level script likelaunch-tmuxlocal.sh which is responsible to read and pass the config file that is then loaded via the root __init__.py. So, all these modules that can be invoked by a script like launch-tmuxlocal.sh depend on the script itself passing the -f arg to the module which will then pass it to the load_config parser invoked in the root __init__.py.

One possible alternative would be to have the explicit HbmpcConfig.load_config() call in each relevant module under the script section (if __name__ == "__main__":) as it is already done in https://github.com/initc3/HoneyBadgerMPC/blob/0914c0269a5d3636b01f40557e8ea432f721bd04/apps/asynchromix/powermixing.py#L167-L170. The inconvenient with this approach would be that one would need to always be careful to add the load_config when writing a new module that requires the configuration step.