sz.h issue with HDF5 filter

[houjun]

Hi,

@rajeeja and I found that when SZ HDF5 filter is enabled, the Intel compiler will complain about not finding sz.h from H5Z_SZ.h when compiling an application code, changing it to #include "sz/sz.h" makes it work.

[robertu94]

How are you compiling and linking with the Intel compiler? Are you using CMake or Autotools? What does your client code look like?

[houjun]

We are using Makefile to link FLASH-X with the SZ HDF5 filter, which calls SZ_errConfigToCdArray() @rajeeja Can you paste the error message if you still have it in your terminal history?

[robertu94]

What would be more helpful would be the makefile you are using. My guess is that you are missing an include flag that either CMake or Make+pkg-config would provide. If it doesn't we can fix that.

[houjun]

The Makefile something like the below, sz is intalled with spack (master branch):

SZ_PATH    = ...
CFLAGS_SZ    = -I${SZ_PATH}/include
FFLAGS_SZ    = -I${SZ_PATH}/include
LIB_SZ    = -L${SZ_PATH}/lib -lhdf5sz -Wl,-rpath,${SZ_PATH}/lib

The code uses SZ as the following:

#include "hdf5_sz/H5Z_SZ.h"

SZ_errConfigToCdArray(&cd_nelmts, &cd_values, ABS, *compression_val, 0, 0, 0);
H5Pset_filter(dataset_plist, H5Z_FILTER_SZ, H5Z_FLAG_MANDATORY, cd_nelmts, cd_values);

Previously we tried using

#include "H5Z_SZ.h"

but the Intel compiler complains H5Z_SZ.h not found, so we changed to hdf5_sz/H5Z_SZ.h and H5Z_SZ.h complains 'sz.h' is not found.

The same code compiles fine with gcc compiler.

[disheng222]

Honestly, I have no clear ideas why Intel compiler doesn't work but gcc works fine. IMHO, if you change H5Z_SZ.h to hdf5_sz/H5Z_SZ.h, maybe you also need to adjust sz.h as well? In fact, I guess this is not needed because sz.h is not put in hfd5_sz directory but based on the SZ's installation path. Since you installed SZ by spack, I don't know where/how you can link SZ's library. If you are using Makefile to build hdf5_sz or so, SZ_PATH should be the installation directory. Could you check if you can find the sz.h in the corresponding path? Since Robert is the key developer of the spack part in SZ, I guess he may give you more hints about linking under spack.

On Wed, Apr 12, 2023 at 5:33 PM Houjun Tang @.***> wrote:

The Makefile something like the below, sz is intalled with spack (master branch):

SZ_PATH = ... CFLAGS_SZ = -I${SZ_PATH}/include FFLAGS_SZ = -I${SZ_PATH}/include LIB_SZ = -L${SZ_PATH}/lib -lhdf5sz -Wl,-rpath,${SZ_PATH}/lib

The code uses SZ as the following:

include "hdf5_sz/H5Z_SZ.h"

SZ_errConfigToCdArray(&cd_nelmts, &cd_values, ABS, *compression_val, 0, 0, 0); H5Pset_filter(dataset_plist, H5Z_FILTER_SZ, H5Z_FLAG_MANDATORY, cd_nelmts, cd_values);

Previously we tried using

include "H5Z_SZ.h"

but the Intel compiler complains H5Z_SZ.h not found, so we changed to hdf5_sz/H5Z_SZ.h and H5Z_SZ.h complains 'sz.h' is not found.

The same code compiles fine with gcc compiler.

— Reply to this email directly, view it on GitHub https://github.com/szcompressor/SZ/issues/104#issuecomment-1506051278, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACK3KSOBKTEPG6RZHLFQIQLXA4UTDANCNFSM6AAAAAAW25IHTY . You are receiving this because you are subscribed to this thread.Message ID: @.***>

[rajeeja]

spack spec sz@master%intel@20.4 +fortran +hdf5 was used to install sz

The sz contribution to HDF5_PLUGIN_PATH is incorrect. It is lib64, but that folder does not exist. Changed to lib. @robertu94

[rajeeja]

spack spec sz@master%intel@20.4 +fortran +hdf5 was used to install sz

The sz contribution to HDF5_PLUGIN_PATH is incorrect. It is lib64, but that folder does not exist. Changed to lib. @robertu94 - same for sz3

[robertu94]

@rajeeja

spack install sz+hdf5
[+] /usr (external cmake-3.22.2-eu4uptjwfrdpp2imsphiw5kfzvqpiuwf)
[+] /usr (external pkgconf-1.8.0-qcja2noiprmroskqkc5z2ddaoh5pzlie)
[+] /home/runderwood/git/contrib/spack/opt/spack/linux-fedora37-icelake/gcc-12.2.1/zlib-1.2.13-qpuezpedu37xahyewd5esz44gbb3kbzh
[+] /home/runderwood/git/contrib/spack/opt/spack/linux-fedora37-icelake/gcc-12.2.1/zstd-1.5.2-pcwlixqybrhnouolrp3vyy2rswxo74hm
[+] /home/runderwood/git/contrib/spack/opt/spack/linux-fedora37-icelake/gcc-12.2.1/hdf5-1.12.2-ibq6o5lvaaw7jpsmb345etk6tylfuxyc
==> Installing sz-2.1.12.2-raau2sdcos65biuk2y76zobsxqiq72on
==> No binary for sz-2.1.12.2-raau2sdcos65biuk2y76zobsxqiq72on found: installing from source
==> Using cached archive: /home/runderwood/git/contrib/spack/var/spack/cache/_source-cache/archive/42/427e263e1fed1b0a56e13e0aff8e6a19c6d78d5f35dd16856876c70ab6066dc6.tar.gz
==> No patches needed for sz
==> sz: Executing phase: 'cmake'
==> sz: Executing phase: 'build'
==> sz: Executing phase: 'install'
==> sz: Successfully installed sz-2.1.12.2-raau2sdcos65biuk2y76zobsxqiq72on
  Stage: 0.05s.  Cmake: 1.65s.  Build: 3.96s.  Install: 0.12s.  Total: 5.83s
[+] /home/runderwood/git/contrib/spack/opt/spack/linux-fedora37-icelake/gcc-12.2.1/sz-2.1.12.2-raau2sdcos65biuk2y76zobsxqiq72on

~ took 19s 
❯ spack cd -i sz+hdf5

linux-fedora37-icelake/gcc-12.2.1/sz-2.1.12.2-raau2sdcos65biuk2y76zobsxqiq72on on  develop [$!] 
❯ ls
include  lib64  share

Interesting. On my system it is set to lib64, but I installed with GCC. Can you please send me the contents of the files printed out by cat $(spack location -i sz+hdf5)/.spack/*? Perhaps there is an environmental variable we should be respecting that we are not? I'll also try installing with the intel compiler and see what happens in that case.

[jared321]

@robertu94 Here is the information for our Intel-based install

> ls $(spack location -i sz@master%intel@20.4)
include  lib  share
> ls $(spack location -i sz@master%intel@20.4)/lib
libhdf5sz.so  libSZ.so

I'll send you the requested log info soon.