Update AMReX, Add pyAMReX, ImpactX Py - Githubissues

radiasoft / download

Install scripts for "curl | bash"

Apache License 2.0

3 stars 5 forks source link

Update AMReX, Add pyAMReX, ImpactX Py #682

Closed ax3l closed 3 months ago

ax3l commented 3 months ago

X-ref: #680 #677 #676

[x] update AMReX to 24.05
- [x] shared build
- [x] add build features needed by ImpactX & WarpX
[x] add pyAMReX 24.05
[x] update ImpactX to 24.05
- [x] depend on external pyAMReX and AMReX

TODO

[x] MPI: do you build sirepo packages with or without MPI? (We can do both, default is with MPI.)
[x] testing (I do not know how)

ax3l commented 3 months ago

attn @moellep @e-carlin @cchall @ncook882

Please feel free to push to this branch as needed, I do not know how to test this :)

e-carlin commented 3 months ago

@ax3l thank you! This looks great. I was actually working on making these changes yesterday. Nice to know I was on the right track:). I'm happy to take it from here. Thanks again!

ax3l commented 3 months ago

Thank you @e-carlin for taking this over 👍

Oh, I see you have a WarpX package as well. That one can reuse the same AMReX and pyAMReX. We could also add a package for openpmd-api for both WarpX and ImpactX to share :)

e-carlin commented 3 months ago

Will do, thanks!

ax3l commented 3 months ago

Thanks!!

For docs, besides the details shared here https://github.com/radiasoft/download/issues/680#issuecomment-2136332327

Let me also share this WarpX doc page on our superbuilds and how to turn them off: https://warpx.readthedocs.io/en/latest/developers/local_compile.html

e-carlin commented 3 months ago

@ax3l whenever I try to install impactx I get the below error. Any idea what would cause this?

[py3;@v2 impactx-24.05]$ codes_cmake2  \
      -DCMAKE_INSTALL_PREFIX="${codes_dir[prefix]}"  \
      -DImpactX_PYTHON=ON \
      -DImpactX_amrex_internal=OFF \
      -DImpactX_pyamrex_internal=OFF
-- The C compiler identification is GNU 12.2.1
-- The CXX compiler identification is GNU 12.2.1
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Could NOT find CCache
CMake Warning (dev) at /usr/share/cmake/Modules/GNUInstallDirs.cmake:380 (message):
  GNUInstallDirs_get_absolute_install_dir called without third argument.
  Using ${dir} from the caller's scope for compatibility with CMake 3.19 and
  below.
Call Stack (most recent call first):
  cmake/ImpactXFunctions.cmake:84 (GNUInstallDirs_get_absolute_install_dir)
  CMakeLists.txt:62 (impactx_set_default_install_dirs)
This warning is for project developers.  Use -Wno-dev to suppress it.

-- Downloading ABLASTR ...
-- ABLASTR repository: https://github.com/ECP-WarpX/WarpX.git (24.05)
-- Searching for pre-installed AMReX ...
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE
-- Found MPI_C: /usr/lib64/mpich/lib/libmpi.so (found version "3.1")
-- Found MPI_CXX: /usr/lib64/mpich/lib/libmpi.so (found version "3.1")
-- Found MPI: TRUE (found version "3.1") found components: C CXX
-- Found OpenMP_CXX: -fopenmp (found version "4.5")
-- Found OpenMP: TRUE (found version "4.5") found components: CXX
-- AMReX: Found version '24.05'
-- Downloading openPMD-api ...
-- openPMD-api repository: https://github.com/openPMD/openPMD-api.git (0.15.2)
-- Found MPI: TRUE (found version "3.1") found components: CXX
-- Using the single-header code from /home/vagrant/src/radiasoft/codes/impactx-20240529.174621/impactx-24.05/build/_deps/fetchedopenpmd-src/share/openPMD/thirdParty/json/single_include/
-- nlohmann-json: Using INTERNAL version '3.9.1'
-- toml11: Using INTERNAL version '3.7.1'
-- Found HDF5: /usr/lib64/mpich/lib/libhdf5.so (found version "1.12.1") found components: C
-- Could NOT find ADIOS2 (missing: ADIOS2_DIR)

openPMD build configuration:
  library Version: 0.15.2
  openPMD Standard: 1.1.0
  C++ Compiler: GNU 12.2.1
    /usr/bin/c++

  Installation: OFF

  Build Type: Release
  Library: static
  CLI Tools: OFF
  Examples: OFF
  Testing: OFF
  Invasive Tests: OFF
  Internal VERIFY: ON
  Build Options:
    MPI: ON
    HDF5: ON
    ADIOS1: OFF
    ADIOS2: OFF
    PYTHON: OFF
    CUDA_EXAMPLES: OFF

-- Performing Test COMPILER_HAS_HIDDEN_VISIBILITY
-- Performing Test COMPILER_HAS_HIDDEN_VISIBILITY - Success
-- Performing Test COMPILER_HAS_HIDDEN_INLINE_VISIBILITY
-- Performing Test COMPILER_HAS_HIDDEN_INLINE_VISIBILITY - Success
-- Performing Test COMPILER_HAS_DEPRECATED_ATTR
-- Performing Test COMPILER_HAS_DEPRECATED_ATTR - Success

WarpX build configuration:
  Version: 24.05 (24.05)
  C++ Compiler: GNU 12.2.1
    /usr/bin/c++

  Installation prefix: /home/vagrant/.local
        bin: bin
        lib: lib
    include: include
      cmake: lib/cmake/WarpX

  Build type: Release
  Build options:
    APP: OFF
    ASCENT: OFF
    COMPUTE: OMP
    DIMS: 3
    Embedded Boundary: OFF
    IPO/LTO: OFF
    LIB: OFF
    MPI: ON
    PARTICLE PRECISION: DOUBLE
    PRECISION: DOUBLE
    PSATD: OFF
    PYTHON: OFF
    OPENPMD: ON
    QED: OFF
    QED table generation: OFF
    QED tools: OFF
    SENSEI: OFF

-- ABLASTR: Using version '' ()
-- Found Python: /home/vagrant/.pyenv/versions/3.9.15/envs/py3/bin/python3.9 (found version "3.9.15") found components: Interpreter Development.Module
-- Downloading pybind11 ...
-- pybind11 repository: https://github.com/pybind/pybind11.git (v2.12.0)
-- pybind11 v2.12.0
-- Found PythonInterp: /home/vagrant/.pyenv/versions/3.9.15/envs/py3/bin/python (found suitable version "3.9.15", minimum required is "3.6")
-- Found PythonLibs: /home/vagrant/.pyenv/versions/3.9.15/lib/libpython3.9.so
-- Performing Test HAS_FLTO
-- Performing Test HAS_FLTO - Success
-- Found MPI: TRUE (found version "3.1") found components: C CXX
-- pyAMReX: Found version '24.05'
-- Found Git: /usr/bin/git (found version "2.40.1")

ImpactX build configuration:
  Version: 24.05 (24.05-nogit)
  C++ Compiler: GNU 12.2.1
    /usr/bin/c++

  Installation prefix: /home/vagrant/.local
        bin: bin
        lib: lib
    include: include
      cmake: lib/cmake/ImpactX
     python: lib/python3.9/site-packages

  Build type: Release
  Build options:
    APP: ON
    COMPUTE: OMP
    IPO/LTO: OFF
    LIB: static
    MPI: ON
    PRECISION: DOUBLE
    PYTHON: ON
    OPENPMD: ON

-- Configuring done (23.5s)
-- Generating done (0.2s)
-- Build files have been written to: /home/vagrant/src/radiasoft/codes/impactx-20240529.174621/impactx-24.05/build
[py3;@v2 impactx-24.05]$

[py3;@v2 impactx-24.05]$ cmake --build build --target install
cmake --build build --target install
[ 49%] Built target openPMD
[ 51%] Built target buildInfopyImpactX
/home/vagrant/src/radiasoft/codes/impactx-20240529.174621/impactx-24.05/build/_deps/fetchedablastr-src/Source/ablastr/parallelization/MPIInitHelpers.cpp:38:23: error: #elif with no expression
   38 | #   elif AMREX_USE_OMP
      |                       ^
gmake[2]: *** [_deps/fetchedablastr-build/CMakeFiles/ablastr_3d.dir/build.make:111: _deps/fetchedablastr-build/CMakeFiles/ablastr_3d.dir/Source/ablastr/parallelization/MPIInitHelpers.cpp.o] Error 1
gmake[1]: *** [CMakeFiles/Makefile2:1548: _deps/fetchedablastr-build/CMakeFiles/ablastr_3d.dir/all] Error 2
gmake: *** [Makefile:146: all] Error 2
[py3;@v2 impactx-24.05]$  all]

ax3l commented 3 months ago

@e-carlin Oh ha, that is a bug. Here is the fix: https://github.com/ECP-WarpX/WarpX/pull/4957

No need to apply it though. Please add -DAMReX_MPI_THREAD_MULTIPLE=ON to the AMReX CMake options, I forgot that option (we want this to be enabled) and once this is on the bug will not be triggered, too.

e-carlin commented 3 months ago

@ax3l thanks! I was able to build and install. Now running into this when I try to use the python bindings. Any ideas?

$ python -c 'import impactx'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/vagrant/.pyenv/versions/py3/lib/python3.9/site-packages/impactx/__init__.py", line 20, in <module>
    from . import impactx_pybind as cxx
ImportError: generic_type: type "ImpactXParIter" referenced unknown base type "amrex::ParIter_impl<amrex::SoAParticle<8, 0>, 8, 0, std::allocator, amrex::DefaultAssignor>"

FWIW - If I build impactx with internal amrex and pyamrex I'm able to successfully import impactx.

e-carlin commented 3 months ago

I think I found the issue. Missing cmake --build build -j 4 --target pip_install for pyamrex.

e-carlin commented 3 months ago

@robnagler please review