By necessity, the architecture of BespokeFit is quite complex and uses a number of packages and patterns that are unfamiliar at least to me, and I suspect also to other potential contributors.
It would be great to flesh out the developer guide with an overview of the structure of the code base, how tasks are passed around, and how tests are organised. I'd be very happy to write this similarly to how we did the user guide; if @jthorton could give dotpoints on the basic structure of the code I can flesh it out into something accessible to everyone. I'd also like to write docstrings for every module (and ideally every function and class).
Some specific questions I have:
The module structure seems to reuse names like "worker", "model", "cache", "app", "base" a lot. What do these terms mean and how are they organised? I can guess most of them but clarity would be good.
The different stages seem to have code in both executor/services/coordinator/stages.py module and in the individual executor/services/*/worker.py. When should code go in one or the other? I gather this corresponds to whether it happens in the coordinator task or gets passed in to its own celery worker?
Could I get a brief summary of the module layout?
How are tests organised? What mocking and harnesses are available? Do we have unit tests, integration tests, any other kinds of tests?
Why does the codebase often provide types that are unions of all the types that share a base type for type hints, rather than just using the base type?
In brief, what would be needed (and in which modules should it be placed) to implement a new optimizer, qcgenerator, fragmenter, or stage?
What else should developers know about BespokeFit?
I am very grateful for the extensive use of type hinting throughout the code base - it makes things much easier to understand :)
Description
By necessity, the architecture of BespokeFit is quite complex and uses a number of packages and patterns that are unfamiliar at least to me, and I suspect also to other potential contributors.
It would be great to flesh out the developer guide with an overview of the structure of the code base, how tasks are passed around, and how tests are organised. I'd be very happy to write this similarly to how we did the user guide; if @jthorton could give dotpoints on the basic structure of the code I can flesh it out into something accessible to everyone. I'd also like to write docstrings for every module (and ideally every function and class).
Some specific questions I have:
executor/services/coordinator/stages.py
module and in the individualexecutor/services/*/worker.py
. When should code go in one or the other? I gather this corresponds to whether it happens in the coordinator task or gets passed in to its own celery worker?I am very grateful for the extensive use of type hinting throughout the code base - it makes things much easier to understand :)