There are two places that the pipeline archives datasets to data library:
in the data sync step to prep for the build
in the publish step after the build has been completed
The data sync step uses the library_archive and library_archive_version bash function and the publish step uses our custom library action. I think this can be standardized, and I think the best way to move forward is for the data sync to use the data library action.
However the data sync may no longer need to use data library if the devcontainer implementation works well, in which case no such upgrade is needed. So the issue will be marked as wontfix for now
There are two places that the pipeline archives datasets to data library:
The data sync step uses the
library_archiveandlibrary_archive_versionbash function and the publish step uses our custom library action. I think this can be standardized, and I think the best way to move forward is for the data sync to use the data library action.However the data sync may no longer need to use data library if the devcontainer implementation works well, in which case no such upgrade is needed. So the issue will be marked as wontfix for now