NWM Predictions: Design Rolling Update Strategy

[rajadain]

Part of #32

[rajadain]

As of #105 we have a notebook that demonstrates converting one NetCDF dataset to Zarr.

What we'd like to do next is come up with a strategy for rolling updates, so that as new NetCDF datasets are encountered, they can be appended to the same Zarr file.

What we want to end up with is something like this:

	0	1	2	3	4	5	6	7	8	9	10	11	12	13	14	15	16	17	18	19	20	21	22	23
0	✅	✅	✅	✅	✅	✅	✅	✅
1		✅	✅	✅	✅	✅	✅	✅	✅
2			✅	✅	✅	✅	✅	✅	✅	✅
3				✅	✅	✅	✅	✅	✅	✅	✅
4					✅	✅	✅	✅	✅	✅	✅	✅
5						✅	✅	✅	✅	✅	✅	✅	✅
6							✅	✅	✅	✅	✅	✅	✅	✅
7								✅	✅	✅	✅	✅	✅	✅	✅
8									✅	✅	✅	✅	✅	✅	✅	✅
9										✅	✅	✅	✅	✅	✅	✅	✅
10											✅	✅	✅	✅	✅	✅	✅	✅
11												✅	✅	✅	✅	✅	✅	✅	✅
12													✅	✅	✅	✅	✅	✅	✅	✅
13														✅	✅	✅	✅	✅	✅	✅	✅
14															✅	✅	✅	✅	✅	✅	✅	✅
15																✅	✅	✅	✅	✅	✅	✅	✅
16																	✅	✅	✅	✅	✅	✅	✅	✅

where the rows are sequential predictions, columns are hours, and cells have a ✅ if there is a prediction for them.

In the above demonstration, each prediction has values for the next 8 hours. For a given hour, there are 8 different predictions of its value. It would be valuable to be able to query how that prediction has evolved over time.

Currently each prediction is its own NetCDF file. By storing all these in one Zarr file, we can make answering the above question easier.

[rajadain]

We may be able to do this via xarray's open_mfdataset.