Current situation with the loader and its YAML template
The history of the loader YAML template is a bit weird, so it may be worth summarising here:
we had the standard_tomo()function in the httomo.data.hdf.loaders module, and the YAML template for the loader was generated from that function
when the block loader was introduced, namely, the StandardTomoLoaderclass in the httomo.loaders.standard_tomo_loader module, we didn't know how to tackle generating a YAML template from the class, so we pushed the problem to the side and simply:
kept httomo.data.hdf.loaders in the YAML template and pipeline
in the refactoring of httomo where the block loader was introduced, we basically treat the very first method in the pipeline as a special case, and thus we do not adhere to the module_path and method config for the loader in the YAML pipeline
The situation we have now is that the YAML config for a loader is inconsistent with where the loader code is actually located (ie, it's in httomo.loaders.standard_tomo_loader rather than httomo.data.hdf.loaders, despite the config implying the latter).
Stuff to address (they're somewhat connected, rather than being totally independent problems)
Loader YAML template generation
Due to the block loader being a class and not a function, enabling the YAML template generator script to work with loaders which are classes and not functions will probably require some tweaks to it. For example, maybe by checking if httomo.loaders is part of the module path, loader config in the pipeline can be caught and then treated differently to methods (which are just functions)?
Referencing a loader in a pipeline file
The old loader (ie, the loader which didn't load individual blocks) in httomo.data.hdf.loaders is not used anymore, yet pipelines all reference httomo.data.hdf.loaders. To be consistent with how methods in a YAML pipeline file are referenced via their:
module path
method function name
it would make sense to have the way to reference an httomo loader in a pipeline to be something like:
Current situation with the loader and its YAML template
The history of the loader YAML template is a bit weird, so it may be worth summarising here:
standard_tomo()function in thehttomo.data.hdf.loadersmodule, and the YAML template for the loader was generated from that functionStandardTomoLoaderclass in thehttomo.loaders.standard_tomo_loadermodule, we didn't know how to tackle generating a YAML template from the class, so we pushed the problem to the side and simply:httomo.data.hdf.loadersin the YAML template and pipelinemodule_pathandmethodconfig for the loader in the YAML pipelineThe situation we have now is that the YAML config for a loader is inconsistent with where the loader code is actually located (ie, it's in
httomo.loaders.standard_tomo_loaderrather thanhttomo.data.hdf.loaders, despite the config implying the latter).Stuff to address (they're somewhat connected, rather than being totally independent problems)
Loader YAML template generation
Due to the block loader being a class and not a function, enabling the YAML template generator script to work with loaders which are classes and not functions will probably require some tweaks to it. For example, maybe by checking if
httomo.loadersis part of the module path, loader config in the pipeline can be caught and then treated differently to methods (which are just functions)?Referencing a loader in a pipeline file
The old loader (ie, the loader which didn't load individual blocks) in
httomo.data.hdf.loadersis not used anymore, yet pipelines all referencehttomo.data.hdf.loaders. To be consistent with how methods in a YAML pipeline file are referenced via their:it would make sense to have the way to reference an httomo loader in a pipeline to be something like: