Wheels: ADIOS2 v2.10.1

[ax3l]

Build wheels with ADIOS2 v2.9.2+, which fixes c-blosc2 issues with null-blocks. Build with 2.10.0, which fixes static builds with ADIOS2 & c-blosc2 on Windows.

Note: since we do not bump the version of the already released 0.15.2 release, this will not create new wheels. Once we publish 0.15.3, this will be included.

Note: ADIOS2 BP5 supports will be available for Windows with the ADIOS 2.10+ release series.

[ax3l]

@vicentebolea can you help me with this all-static deployment of ADIOS 2.9.2? It is related to the upstream issues I opened :)

[ax3l]

Does BP5 not work on Windows for ADIOS2 2.9.2?

# BP5
if(ADIOS2_USE_BP5 AND NOT WIN32)
  set(ADIOS2_HAVE_BP5 TRUE)
endif()

We want to switch our default output to BP5 after our last discussion w/ @pnorbert :) Serial read support would be awesome on any platform :)

Update: confirmed, will be part of the ADIOS2 v2.10 release series (already in ADIOS2 development branch).

[ax3l]

ADIOS 2.10.0-rc1 on Windows:

2024-01-30T18:43:37.2016284Z   adios2_c_variable.cpp
2024-01-30T18:43:37.4167988Z LINK : fatal error LNK1104: cannot open file 'ffi.lib' [D:\a\openPMD-api\openPMD-api\src\build-adios2\source\utils\bpls.vcxproj]
2024-01-30T18:43:37.4954992Z   main.cpp
2024-01-30T18:43:38.2227279Z   Reorganize.cpp
2024-01-30T18:43:38.2323129Z   adios2_c_attribute.cpp
2024-01-30T18:43:38.5659633Z   IO.cpp
2024-01-30T18:43:39.1746622Z   adios2_c_operator.cpp
2024-01-30T18:43:39.4554626Z   Utils.cpp
2024-01-30T18:43:39.6007589Z   Operator.cpp
2024-01-30T18:43:39.9410487Z   Generating Code...
2024-01-30T18:43:40.0518750Z   Generating Code...
2024-01-30T18:43:40.3069063Z LINK : fatal error LNK1104: cannot open file 'ffi.lib' [D:\a\openPMD-api\openPMD-api\src\build-adios2\source\utils\adios_reorganize.vcxproj]

https://github.com/ornladios/ADIOS2/issues/4019

[franzpoeschel]

I think I found the problem and it looks like a CMake bug. I was able to reproduce the same issue with CMake 3.23 on Windows (install Blosc2 and ADIOS2 statically --> try configuring openPMD-api with these dependencies). I checked the CMake scripts and they looked fine, so instead I upgraded to CMake 3.28, and that seems to work now.

[ax3l]

Awesome!

Hm, CI installs CMake 3.28.1 already... Let me check if it picks up the right version.

[vicentebolea]

@ax3l it appears that we were exporting blosc2 unnecessarily, and when doing out find_dependency defaulted to module as per cmake 3.24, rather than config. Solution seems to be not to export it anyways as our bloscs2 dep is private.

[ax3l]

Thank you, @vicentebolea!

Is there potentially a patch I can apply for this? Note that I am doing an all-static build for pip here, so to some extent I think we need to export blosc2 at least on the link lines.

[vicentebolea]

@ax3l we are release adios 2.10 (final release) very soon. Otherwise I think that those changes are patchable (in 2.10 and 2.9.2)

[ax3l]

@vicentebolea Awesome - can I test a branch / RC here? :)

[ax3l]

@vicentebolea Can I test a branch / RC for ADIOS2 2.10 already? :)

[vicentebolea]

@vicentebolea Can I test a branch / RC for ADIOS2 2.10 already? :)

Current release_210 should have the fix for the windows static build

[ax3l]

@vicentebolea thanks, I pushed to use release_210 and still see in an all-static build:

• c-blosc2 static
• adios2 static
• openPMD-api static
• openPMD-api Python module: shared

the same issue.

Snippet from the Windows CI in this PR:

2024-03-12T18:13:01.6247052Z   CMake Error at C:/Program Files (x86)/ADIOS2/lib/cmake/adios2/adios2-targets.cmake:79 (set_target_properties):
2024-03-12T18:13:01.6248634Z     The link interface of target "adios2::core" contains:
2024-03-12T18:13:01.6251135Z 
2024-03-12T18:13:01.6253673Z       Blosc2::blosc2_static
2024-03-12T18:13:01.6255673Z 
2024-03-12T18:13:01.6258466Z     but the target was not found.  Possible reasons include:
2024-03-12T18:13:01.6260527Z 
2024-03-12T18:13:01.6263217Z       * There is a typo in the target name.
2024-03-12T18:13:01.6265554Z       * A find_package call is missing for an IMPORTED target.
2024-03-12T18:13:01.6267457Z       * An ALIAS target is missing.
2024-03-12T18:13:01.6269395Z 
2024-03-12T18:13:01.6271796Z   Call Stack (most recent call first):
2024-03-12T18:13:01.6274492Z     C:/Program Files (x86)/ADIOS2/lib/cmake/adios2/adios2-config-common.cmake:198 (include)
2024-03-12T18:13:01.6277493Z     C:/Program Files (x86)/ADIOS2/lib/cmake/adios2/adios2-config.cmake:29 (include)
2024-03-12T18:13:01.6278618Z     CMakeLists.txt:443 (find_package)

[vicentebolea]

@ax3l can you try the 2.10.0-rc1 and when find_package(ADIOS2) add a CONFIG.

[vicentebolea]

Try this branch for fixing the windows build: https://github.com/ornladios/ADIOS2/pull/4093