Currently (0.14.1) the set of tests is baked into the library. There is no convenient way of customizing which tests are run, what they do, or to extend the set of tests with additional tests. If we want to facilitate this, we should consider refactoring the current structure into a plugin-based architecture.
The base howfairis library should then offer the infrastructure to run tests and report on their results. The tests themselves should come from a plugin (i.e. from a separate library, not necessarily made by us). Under this scenario, howfairis should have a loader to discover what plugins are available on the user's system / in their environment. It should also provide an abstract base class for a check, AbstractCheck, which plugins can then implement. The base class should probably have a property (to be overridden in the plugin's implementation) defining in which of the 5 categories the Check's results should be reported (TBD: are there any Checks that fall outside of the 5 categories?)
To help rate limiting, it is probably helpful if Check's __init__ can take some commonly required data as its argument, for example the contents of the readme.
Currently (0.14.1) the set of tests is baked into the library. There is no convenient way of
customizing which tests are run, what they do, or to extend the set of tests with additional tests. If we want to facilitate this, we should consider refactoring the current structure into a plugin-based architecture.The base
howfairislibrary should then offer the infrastructure to run tests and report on their results. The tests themselves should come from a plugin (i.e. from a separate library, not necessarily made by us). Under this scenario,howfairisshould have a loader to discover what plugins are available on the user's system / in their environment. It should also provide an abstract base class for a check,AbstractCheck, which plugins can then implement. The base class should probably have a property (to be overridden in the plugin's implementation) defining in which of the 5 categories the Check's results should be reported (TBD: are there any Checks that fall outside of the 5 categories?)Tests like https://github.com/fair-software/howfairis/blob/008552b7266e229bd38553631d7dfe3554df18b2/howfairis/mixins/citation_mixin.py#L40-L51 should be moved into a separate plugin library, maybe something like
howfairis-nlescTo help rate limiting, it is probably helpful if
Check's__init__can take some commonly required data as its argument, for example the contents of the readme.Refer to https://packaging.python.org/en/latest/guides/creating-and-discovering-plugins/ for more information on various plugin discovery methods.