Move _compose_module to BasePipelineWriter, since it does not modify the Artifact Collection but returns a string representation that can be parametrized which is more suitable for Writer classes.
generate_module_text refactored out. Sorting session artifacts is now done in constructor and prettify is in _compose_modules. All external references replaced with _compose_module.
Move get_module, which requires a written version of the module to a utility function and rename to load_as_module in lineapy.plugins.loader. Since the desired use case is to run a function/module, this should be replaced by proper execution of a module through the LineaGraph in the future.
Cleanup constructor for ArtifactCollection. Grouping artifacts by session is now refactored into a helper function _get_artifacts_grouped_by_session and applied to both target and reuse-precomputed artifact lists.
Added Todos add better checks to validate reuse-precomputed artifacts by their Session to keep SessionArtifact contained to only operate within a single Session as much as possible.
sort_session_artifacts broken down into validate_dependencies helper and create_inter_session_taskgraph before sorting the session graph.
Changed API for ArtifactCollection, which is internal, to only accept LineaArtifactDefs, and move the conversion to ArtifactDefs to layer between user facing UI before creation of ArtifactCollection.
Other minor changes with naming, deleting unused variables, type hinting, fixing tests to use newly refactored methods.
Description
Refactor internal Artifact Collection class.
_compose_moduleto BasePipelineWriter, since it does not modify the Artifact Collection but returns a string representation that can be parametrized which is more suitable for Writer classes.generate_module_textrefactored out. Sorting session artifacts is now done in constructor and prettify is in_compose_modules. All external references replaced with_compose_module.get_module, which requires a written version of the module to a utility function and rename toload_as_moduleinlineapy.plugins.loader. Since the desired use case is to run a function/module, this should be replaced by proper execution of a module through the LineaGraph in the future._get_artifacts_grouped_by_sessionand applied to both target and reuse-precomputed artifact lists.sort_session_artifactsbroken down intovalidate_dependencieshelper andcreate_inter_session_taskgraphbefore sorting the session graph.Fixes
LIN-656, Refactor Artifact Collection
Type of change
How Has This Been Tested?
Run all current pytests.