Update read_processing_set and load_processing_set to be compatible with cloud-based storage (such as S3). - Githubissues

casangi / xradio

Xarray Radio Astronomy Data IO

Other

9 stars 5 forks source link

Update read_processing_set and load_processing_set to be compatible with cloud-based storage (such as S3). #140

Closed Jan-Willem closed 2 months ago

Jan-Willem commented 4 months ago

The following functions should be updated so that data can be accessed in cloud-based storage:

https://github.com/casangi/xradio/blob/main/src/xradio/vis/load_processing_set.py (zarr based access)
https://github.com/casangi/xradio/blob/main/src/xradio/vis/read_processing_set.py (xarray(dask) based access)

The small dataset Antennae_North.cal.lsrk.split.vis.zarr can be used for developing the code and testing.

How to obtain the dataset is explained in this tutorial:

https://github.com/casangi/xradio/blob/main/doc/vis_tutorial.ipynb