Right now the data uploading is handled by zarr.copy_store. This works well when we have good internet connection, but for the ship pipeline it may be better to use rclone, so that failed stores can be easily over written, and that it is easy to copy child folders also in one go.
Right now the data uploading is handled by
zarr.copy_store. This works well when we have good internet connection, but for the ship pipeline it may be better to userclone, so that failed stores can be easily over written, and that it is easy to copy child folders also in one go.