azavea / noaa-hydro-data

NOAA Phase 2 Hydrological Data Processing
11 stars 3 forks source link

Determine reason for size difference of rechunked retrospective Zarr #133

Open jpolchlo opened 1 year ago

jpolchlo commented 1 year ago

We produced a rechunked version of the NWM retrospective data set, geared toward time-series queries. This new Zarr file (s3://azavea-noaa-hydro-data/experiments/jp/rechunk/output.zarr/) is half the size of the original (s3://noaa-nwm-retrospective-2-1-zarr-pds/chrtout.zarr). We ought to take a moment to understand why this happened.