mpiannucci
commented
2 years ago Current Downsides to this approach:
- aforementioned 404 errors
- The metadata is computed for the entire tree no matter if you need it or not, should be lazy computed since groups can be loaded on their own
- tile size should be dynamic
- the reprojection requires the entire dataset to be loaded because rioxarray can't do this in parallel or lazily right now (as far as I know) maybe dask can be smart about this. In general, we need to be able to only grab data chunks needed for each tiles reprojection. Not sure anything can do this out of the box right now
- non xy chunk shape dimensions are hard coded to the source datasets chunk sizes
This creates a POC working zarr tree API. It is currently hard coded to 256 pixel tiles, and uses
rioxarrayto reproject under the hood. There is a working example to get the map representation of ww3 and plot it at level 0 inxpublish/test_get_chunk.ipynb.One thing that is not clear right now is why fetching the dataset is so slow, it seems to hang because of this failed request, which we dont handle cuz im not sure what should be returned for them. But getting the data works!!!! Enough to start work on a javascript client test :)