Should dodal have a defined API?

[callumforrester]

Part of the reason for GDA's maintenance difficulties is that every piece of code in GDA is accessible via Jython, including classes that were only ever meant to be internal and disposable. This makes every change a potentially breaking change and means that the whole codebase is, in effect, the API. It has been a source of huge amounts of technical debt.

As dodal gets bigger, I am concerned that this may happen here too. As the number of exposed functions gets larger, the scale of changes needed when we refactor increases and the more fragile the system becomes. Some examples:

• https://github.com/DiamondLightSource/dodal/pull/304#pullrequestreview-1898237011
• https://github.com/DiamondLightSource/dodal/issues/305
• https://github.com/DiamondLightSource/dodal/pull/280#discussion_r1445858621
• https://github.com/DiamondLightSource/dodal/pull/193#discussion_r1352506583

With hindsight, we wish we had made a "Jython API" in GDA, i.e. a defined set of classes/functions that were exposed to Jython (and no Jython access to any internals). We would only have had to worry about breaking changes to the API functions, and refactoring would become easier.

Making this issue to discuss whether dodal could benefit from something similar. If so what would it look like? Could we make use of __all__? Would the API just consist of "beamlines", plans and a few util functions? Is there actually enough non-API code to justify it?

A few people whom I think might have opinions and/or can expand further on the Jython issue, feel free to tag others: @DiamondJoseph @DominicOram @olliesilvester @tpoliaw @coretl @dperl-dls @phathaway

Acceptance Criteria

• Dodal functioning with "hidden" code that isn't exposed to the outside world and be refactored without breaking changes
• API structure defined in documentation so it can be reproduced with more beamline adoption.

[olliesilvester]

Overall I definitely think this is a good idea.

Would the API just consist of "beamlines", plans and a few util functions? Is there actually enough non-API code to justify it?

It depends on whether this API is for the purpose of wanting to add devices to dodal, or wanting to create plans using dodal devices, or both. If both, then the API would contain the majority of dodal's codebase: All devices and their functions (specifically the functions which plans should be able to access), utility functions, and the stuff in beamlines.

Regardless, it will still be useful to be thinking about this kind of thing as we add more devices. Also @rtuck99 might have some thoughts

[DominicOram]

I agree that a well defined API will be needed at some point. I would probably argue though that usage statistics are a higher priority. That can then guide what the API is.