Adios2 using Blosc2

[titoiride]

Is there a reason why in the install_gpu_dependencies.sh script for Perlmutter, c-blosc is recommended instead of c-blosc2?

https://github.com/ECP-WarpX/WarpX/blob/f9f3db5534836b5ddc53b10310007522bf84e9cd/Tools/machines/perlmutter-nersc/install_gpu_dependencies.sh#L50-L64

I'm trying to understand if this is related to an error I got writing files

Warning: ADIOS2 backend does not support compression method blosc. Continuing without compression.
Original error: [Tue Jun 11 14:00:10 2024] [ADIOS2 EXCEPTION] <Operator> <OperatorFactory> <MakeOperator> : ADIOS2 didn't compile with blosc library, operator not added

that I generated using these options

fulldiag.coarsening_ratio = 16 16 16
fulldiag.write_species = 1
fulldiag.format = openpmd
fulldiag.openpmd_backend = bp
fulldiag.openpmd_encoding = g
fulldiag.adios2_operator.type = blosc
fulldiag.adios2_operator.parameters.compressor = zstd
fulldiag.adios2_operator.parameters.clevel = 1
fulldiag.adios2_operator.parameters.doshuffle = BLOSC_BITSHUFFLE
fulldiag.adios2_operator.parameters.threshold = 2048
fulldiag.adios2_operator.parameters.nthreads = 6  # per MPI rank (and thus per GPU)

I always used the recommended install script for dependencies, but this is the first time I actually use Adios2 outputs, I've always used HDF5 before, so I may not have noticed problems in the past. Mine is mainly a curiosity, I noticed that even the c-blosc repository now recommends using c-blosc2 and the recommended cmake flag for Adios (-DADIOS2_USE_Blosc=ON)

https://github.com/ECP-WarpX/WarpX/blob/f9f3db5534836b5ddc53b10310007522bf84e9cd/Tools/machines/perlmutter-nersc/install_gpu_dependencies.sh#L76

doesn't seem to appear in the adios build folder using ccmake, although I can see -DADIOS2_USE_Blosc2=AUTO. At the end of the compilation, Adios flags that it is in fact not built with blosc support, as per the warning message.

[titoiride]

As an extra note, this is the final Adios2 build configuration that I see using the recommended dependency installation (i.e., c-blosc)

ADIOS2 build configuration:
  ADIOS Version: 2.10.1
  C++ Compiler : GNU 12.3.0 CrayPrgEnv
    /opt/cray/pe/craype/2.7.30/bin/CC

  Installation prefix: /global/cfs/cdirs/m558/terzani/sw/perlmutter/gpu/adios2
        bin: bin
        lib: lib64
    include: include
      cmake: lib64/cmake/adios2

  Features:
    Library Type: shared
    Build Type:   Release
    Testing: OFF
    Examples: OFF
    Build Options:
      DataMan            : OFF
      DataSpaces         : OFF
      HDF5               : ON
      HDF5_VOL           : OFF
      MHS                : ON
      SST                : ON
      Fortran            : OFF
      MPI                : ON
      Python             : OFF
      PIP                : OFF
      Blosc2             : OFF
      BZip2              : ON
      LIBPRESSIO         : OFF
      MGARD              : OFF
      MGARD_MDR          : OFF
      PNG                : ON
      SZ                 : OFF
      ZFP                : OFF
      DAOS               : OFF
      IME                : OFF
      O_DIRECT           : ON
      Sodium             : OFF
      Catalyst           : OFF
      SysVShMem          : ON
      UCX                : OFF
      ZeroMQ             : OFF
      Profiling          : ON
      Endian_Reverse     : OFF
      Derived_Variable   : OFF
      AWSSDK             : OFF
      GPU_Support        : OFF
      CUDA               : OFF
      Kokkos             : OFF
      Kokkos_CUDA        : OFF
      Kokkos_HIP         : OFF
      Kokkos_SYCL        : OFF
      Campaign           : ON
    Possible RDMA DataPlanes for SST:  fabric

[ax3l]

Yes, C-Blosc2 is not yet available with the ADIOS2 version we install (2.8.3). I will bump ADIOS2 and C-Blosc to C-Blosc2 soon in all docs to keep them consistent. We had a couple of regressions when we implemented C-Blosc2, which are now solved in the latest ADIOS2 releases.

I am surprised you see Blosc2 already showing up in 2.8.3 in your last comment - was that that version of ADIOS2 or a newer one?

[titoiride]

Sorry, you're right, I made a mistake reporting. I modified the dependency script to checkout the latest available tags:

  git clone https://github.com/ornladios/ADIOS2.git ${SW_DIR}/adios2
  cd ${SW_DIR}/adios2
  git fetch --tags
  latestTag=$(git describe --tags `git rev-list --tags --max-count=1`)
  git checkout $latestTag

which makes me think is why I see the discrepancy. I have version 2.10.1.