This adds a write_to_remote option that if enabled, writes the output directly to cloud storage (s3 or gcs)
It uses a touch file as the output instead of the zarr folder, with the uri passed separately as a param. This is so that an FSStore can be instantiated in the snakemake rule, instead of generating locally then copying.
Had to add a final() function around the outputs that are optionally on remote, and this function adds the remote prefix, along with applying the storage() function.
To do:
[x] add support for s3 as well (currently only gcs)
[x] only the tif_stacks_to_ome_zarr script is updated, will need to also update the zarr_to_ome_zarr script similarly
This adds a
write_to_remote
option that if enabled, writes the output directly to cloud storage (s3 or gcs)It uses a touch file as the output instead of the zarr folder, with the uri passed separately as a param. This is so that an FSStore can be instantiated in the snakemake rule, instead of generating locally then copying.
Had to add a
final()
function around the outputs that are optionally on remote, and this function adds the remote prefix, along with applying the storage() function.To do: