Not every ingest, especially some of the older ones, indicates where the quasi-forever home of the output data is. Some just indicate that you should shoomp it over to Apollo and ingest it. It'd be good to connect the processing to the data stashing and indicate either a shell command with a destination, e.g., /workspace/Shared/Tech_Projects/Arctic_EDS/project_data/rasdaman_datasets/design_freezing_index or as we've done more recently, have a function within the notebook to copy data to a backed up place. Here is an example:
if COPY_OUTPUTS_TO_ARCHIVE:
archive_fps = [archive_path / ''.join(x.name) for x in output_fps]
print(f"Copying {len(archive_fps)} files to {archive_path}...")
for src, dst in zip(output_fps, archive_fps):
shutil.copy(src, dst)
assert(len([x for x in archive_path.rglob("*.tif")]) == len(output_fps))
else:
print("No files were copied from the project output directory to the archive directory.")
Not every ingest, especially some of the older ones, indicates where the quasi-forever home of the output data is. Some just indicate that you should shoomp it over to Apollo and ingest it. It'd be good to connect the processing to the data stashing and indicate either a shell command with a destination, e.g.,
/workspace/Shared/Tech_Projects/Arctic_EDS/project_data/rasdaman_datasets/design_freezing_indexor as we've done more recently, have a function within the notebook to copy data to a backed up place. Here is an example: