openclimatefix / Satip

Satip contains the code necessary for retrieving, transforming and storing EUMETSAT data
https://satip.readthedocs.io/
MIT License
41 stars 28 forks source link

Ability to save to different sinks in app.py #235

Closed vikasgrewal16 closed 6 months ago

vikasgrewal16 commented 6 months ago

Pull Request

Description

This PR achieve the flexibility of saving to different backends using fsspec in your Python application i have modified the save_to_zarr_to_s3 function in the utils.py file to accept a parameter indicating the desired backend

Fixes #

Example Usage

save_to_zarr(my_dataset, 's3://my-bucket/my-data.zarr', backend='s3') save_to_zarr(my_dataset, 'file:///path/to/local/data.zarr', backend='file') save_to_zarr(my_dataset, 'gcs://my-bucket/my-data.zarr', backend='gcs')

Checklist:

jacobbieker commented 6 months ago

Thank you for doing this work! And it looks good! Unfortunately, https://github.com/openclimatefix/Satip/pull/233 has already implemented the same functionality.

jacobbieker commented 6 months ago

Closing this as the underlying issue was fixed in #233