Open cpelley opened 1 month ago
CLIs are all available in a flat space namespace. That is, all under
improver.cli
Similarly, I think it might be useful to make plugins available under a flat namespace (thereby protecting science configurations from changes in project layout/structure). For example, perhapsimprover.plugin.Standardise
.This doesn't mean the plugins themselves need live under a root subpackage folder, instead they could simply be made available through a single namespace.
At the time I was playing with https://github.com/metoppv/improver/pull/1654, I had an idea of adding improver.api
as a way of marking what is intended as public interface. At the time that was for CLI-generating functions, but could be instead for other interfaces.
# improver/api/__init__.py
"""IMPROVER public API."""
from importlib import import_module
def __getattr__(name):
mod = import_module(f"improver.cli.{name}")
return mod.process
Interesting idea š One downside is discoverability. You wont be able to query what there is, only try asking for it and seeing if it fails Also, at least currently, plugins are defined all over the shop in IMPROVER. The main benefit I see of this it not having to import all the plugins into the 1 namespace (which I'm not sure has much of a performance concern anymore if everything runs within a single python invocation). There could be other benefits that I haven't thought of mind :)
...plugins are defined all over the shop in IMPROVER...
š±
TBH, it's numpy that take most of the import hit so importing 1 processing module is the for the most part the same as importing them all.
As part of the https://github.com/metoppv/improver/issues/1998 work on moving CLI logic to the plugin layer to ultimately access plugins directly rather than via the CLI, this open up a necessary plication right now that plugins when called from where they currently live, would make users subject to changes in the layout of the IMPROVER repository.
CLIs are all available in a flat space namespace. That is, all under
improver.cli
Similarly, I think it might be useful to make plugins available under a flat namespace (thereby protecting science configurations from changes in project layout/structure). For example, perhapsimprover.plugin.Standardise
.This doesn't mean the plugins themselves need live under a root subpackage folder, instead they could simply be made available through a single namespace.
To illustrate (not proposed solution):
Issues