Get used to LibPressio by measuring overhead

The goal of this task is to estimate the runtime and memory overhead of libpressio for several different datasets and configurations. The output would be most useful as a CSV file.

Some datasets to consider:

Hurricane
CESM
Scale LETKF
HACC
NYX
Mirranda
S3D

Some configurations to consider:

For each pick a few different levels near .01, 1, 2, 5, and 30% of the value range of the dataset

SZ with ABS
ZFP with accuracy mode (precision mode and fixed rate mode are nice to have, but trickier to configure)
MGARD (if it doesn't crash) "l-infinity" mode (just set tolerance)

Some timing/measurement tools:

/usr/bin/time getrusage -- used by the memory metric in libpressio posix_gettime used by the time metric in libpressio

please ensure that robertu94_packages spack repo is up to date (git pull). It has a new version of libpressio and a new version of zfp that enable the utitlies to be built as shown below.

sz -- you can install this via spack install sz sz -z -i CLOUDf48 -M ABS -A 1e-6 -f -3 500 500 100 this does compression sz -x -f -M ABS -A 1e-6 -3 500 500 100 -s CLOUDf48.sz this does decompression

zfp -- you can install this via spack install zfp@develop+utilities

zfp -i CLOUDf48 -z CLOUDf48.zfp -3 100 500 500 -f -a 1e-6 compress data `zfp -z CLOUDf48.zfp - o CLOUDf48.dec -3 100 500 500 -f -a 1e-6" decompress data

note that the dimension information flips when using ZFP. Use the ordering I used for SZ when using libpressio.

You might also find spack install libpressio-tools ^libpressio+sz+zfp useful.

pressio -i CLOUDf48 -m time -M all -d 500 -d 500 -d 100 -t float sz -o sz:abs_err_bound=1e-6 -o sz:error_bound_mode_str=abs

pressio -i CLOUDf48 -m time -M all -d 500 -d 500 -d 100 -t float zfp -o zfp:accuracy=1e-6

robertu94 / libpressio-predict

Get used to LibPressio by measuring overhead #6