noaa-oar-arl / monetio

The Model and ObservatioN Evaluation Tool I/O package
https://monetio.readthedocs.io
MIT License
16 stars 19 forks source link

Add RRFS-SD reader #154

Open bbakernoaa opened 6 months ago

bbakernoaa commented 6 months ago

create a separate reader for RRFS-SD. This is based on _rrfs_cmaq_mm but removes many of the functions as rrfs-sd is obviously much simpler.

bbakernoaa commented 6 months ago

this addresses #152

zmoon commented 6 months ago

@bbakernoaa I made an extraction (forecast hour 6 of the run you shared, 5 levels closest to surface, selected variables, etc.), now available at https://csl.noaa.gov/groups/csl4/modeldata/melodies-monet/data/example_model_data/rrfssd_example/rrfs-sd_dynf006.nc (104M) for testing.

We can set it up like I did here, which also uses data stored in that location.

bbakernoaa commented 6 months ago

@bbakernoaa I made an extraction (forecast hour 6 of the run you shared, 5 levels closest to surface, selected variables, etc.), now available at https://csl.noaa.gov/groups/csl4/modeldata/melodies-monet/data/example_model_data/rrfssd_example/rrfs-sd_dynf006.nc (104M) for testing.

We can set it up like I did here, which also uses data stored in that location.

maybe we can add some netcdf tricks to compress that even further using integers instead of floats and the add_offset and scale_factor netcdf attributes

zmoon commented 6 months ago

maybe we can add some netcdf tricks to compress that even further using integers instead of floats and the add_offset and scale_factor netcdf attributes

Perhaps, but I think it is better to keep format closer to the original. And I used NCO lossy compression, which already decreases the size a lot due to quantization, I don't know if additionally doing the int packing transform would make it any more compressible.