NCAR / ParallelIO

A high-level Parallel I/O Library for structured grid applications
Apache License 2.0
134 stars 52 forks source link

Add support for new netCDF functions for lossy compression: nc_def_var_quantize/nc_inq_var_quantize #1890

Open edwardhartnett opened 2 years ago

edwardhartnett commented 2 years ago

We have added quantization with the bitgroom algorithm to the netCDF C (and Fortran) library. So now users can specify a required number of significant digits for a variable, and the excess bits are (in alternating floats) set to 0 and 1. In this way, the data compresses a lot better.

The combination of quantization and compression yields lossy compression.

This capability should be added to PIO.