This PR allows Earthmover projects to be "composable", allowing developers to import and build on existing projects/packages/bundles.
Tests and QC done:
I tested this on a Jeffco NWEA-MAP Earthmover project and matched the result to a past run of a locally modified version of the NWEA-MAP bundle. I did further testing of multiple packages, macro handling, and relative paths using this test bundle
earthmover_composition_testing.zip
Future ToDos and Questions:
We currently use a BUNDLE_DIR parameter to specify relative paths in most bundles. For those bundles to be used as packages, any filepaths will need to be either relative to the location of the earthmover.yaml file (ex. ./seeds/assessments.csv with our typical folder structure) or entirely specified as parameters similar to how INPUT_FILE is typically handled now. This change will need to be made in order for current bundles to be used as packages.
There are some limitations/considerations noted in the readme around being aware that the ordering of packages has an impact on the order of the merge, with later packages overwriting earlier ones. This would only affect projects where two packages are overwriting the same node. Namespacing macros is also recommended to make sure the correct macro is applied to each package.
The function that handles replacing relative filepaths in the package yaml (package.load_package_yaml()) works but will need to be updated if there are future changes to earthmover that would allow filepaths to be provided in new operations or under different keys. It could be rewritten to be more robust.
Description & motivation
This PR allows Earthmover projects to be "composable", allowing developers to import and build on existing projects/packages/bundles.
Tests and QC done:
I tested this on a Jeffco NWEA-MAP Earthmover project and matched the result to a past run of a locally modified version of the NWEA-MAP bundle. I did further testing of multiple packages, macro handling, and relative paths using this test bundle earthmover_composition_testing.zip
Future ToDos and Questions:
BUNDLE_DIRparameter to specify relative paths in most bundles. For those bundles to be used as packages, any filepaths will need to be either relative to the location of theearthmover.yamlfile (ex../seeds/assessments.csvwith our typical folder structure) or entirely specified as parameters similar to howINPUT_FILEis typically handled now. This change will need to be made in order for current bundles to be used as packages.package.load_package_yaml()) works but will need to be updated if there are future changes to earthmover that would allow filepaths to be provided in new operations or under different keys. It could be rewritten to be more robust.