MetOffice / tcd-XIOS-demonstration

Demonstration code and examples for XIOS (XML I/O Server) multi-version use
BSD 3-Clause "New" or "Revised" License
0 stars 5 forks source link

NetCDF test for quantization and compression #29

Open harry-shepherd opened 1 month ago

harry-shepherd commented 1 month ago

NetCDF quantization, along with gzip compression provides an interesting approach to reducing the data output from models. This is not currently implemented in XIOS. This issue aims to construct a small standalone test to demonstrate the functionality in NetCDF. The aim is to:

As there are three different quantization methods; bit grooming, granular bit round, and bit round, and I will explore all three.

https://docs.unidata.ucar.edu/netcdf-c/current/md__media_psf_Home_Desktop_netcdf_releases_v4_9_2_release_netcdf_c_docs_quantize.html#:~:text=The%20quantize%20algorithms%20assist%20with%20lossy%20compression%20by,built%20into%20netCDF-4%2C%20to%20better%20compress%20the%20data.

harry-shepherd commented 1 month ago

The example will be written in C, as this is more directly relevant to the C++ implementation in XIOS.