search

casangi / xradio

Xarray Radio Astronomy Data IO
Other
9 stars 5 forks source link

load_image function is creating a DAG when opening a img.zarr file. #145

Closed Jan-Willem closed 4 months ago

Jan-Willem commented 4 months ago

The load_image function (just like load_processing_set) should load data directly into memory (using Zarr) and not make use of Xarray-backed Dask (which would cause a DAG within a DAG when using the function inside AstroVIPER nodetasks).

Code example:

from xradio.image import load_image, read_image
feather_xds = load_image("feathered_image.img.zarr")

The bokeh dashboard:

Screenshot 2024-03-05 at 4 40 12 PM

It is expected that the bokeh dashboard should be empty.

dmehring commented 4 months ago

@Jan-Willem would you like to test my fix on this branch before I merge this branch?

Jan-Willem commented 4 months ago

@dmehring I tested the branch and load_image no longer triggers Dask compute. Thanks.