search

multiphenics / multiphenicsx

multiphenicsx - easy prototyping of multiphysics problems in FEniCSx
https://multiphenics.github.io/
GNU Lesser General Public License v3.0
39 stars 7 forks source link

move away from jit compilation of the C++ backend #16

Closed francesco-ballarin closed 10 months ago

francesco-ballarin commented 10 months ago

https://github.com/multiphenics/multiphenicsx/issues/15 has shown that jit compilation with cppimport can be fragile on HPC systems.

At the same time, when upstream PR https://github.com/FEniCS/dolfinx/pull/2820 will get merged, we will need to rewrite wrappings of C++ backend to use nanobind instead of pybind11. That might be a good time to consider moving away from cppimport, because it seems to only support pybind11 and not yet nanobind. In that case, the C++ backend should be compiled when running pip install.

Keep an eye out for https://github.com/FEniCS/dolfinx/pull/2707 as well, since it may show how to do this using pyproject.toml only.

An added bonus could be to have both options available:

  1. if the library was pip installed-ed, then C++ backend and its wrappers were already compiled: just use them. This would be suggested for end users.
  2. if the source tree is in the python path, and therefore the C++ library is not compiled yet, use jit. This could be helpful for developers.
francesco-ballarin commented 10 months ago

Hi @hermanmakhm, I am working on this, in particular getting rid of cppimport, in https://github.com/multiphenics/multiphenicsx/tree/pyproject-cmake

Can you create a new conda environment on your HPC system (I wouldn't want you to mess up the existing one) and install multiphenicsx as follows:

python3 -m pip install scikit-build-core[pyproject]
python3 -m pip install --check-build-dependencies --no-build-isolation .[tutorials]

and tell me if you get import issues as in https://github.com/multiphenics/multiphenicsx/issues/15, including cases with few, moderate (< 220) and a lot (> 220) processors?

Note that the installation instructions become a bit more complex (the website will be updated to reflect this), but that will be the case of dolfinx as well once https://github.com/FEniCS/dolfinx/pull/2707 is merged.

hermanmakhm commented 10 months ago

I tried to run the pip install --check-build-dependencies --no-build-isolation .[tutorials] by installing fenicsx via conda-forge but:

ERROR: Some build dependencies for file:///tmp_user/sator/hmmak/multiphenicsx conflict with the backend dependencies: fenics-dolfinx==0.7.0 is incompatible with fenics-dolfinx >=0.8.0.dev0, <0.9.0.

I also tried running this shell script to install dolfinx at the current main but weirdly pip install of dolfinx started to give me this error (which hasn't happened before). Do you know how to fix it? Or would I need to raise an issue with dolfinx?

Processing /tmp_user/sator/hmmak/build-tmp/dolfinx/python
  Installing build dependencies ... error
  error: subprocess-exited-with-error

  × pip subprocess to install build dependencies did not run successfully.
  │ exit code: 1
  ╰─> [160 lines of output]
      Collecting pybind11>=2.9.1
        Using cached pybind11-2.11.1-py3-none-any.whl.metadata (9.5 kB)
      Collecting petsc4py
        Using cached petsc4py-3.20.0.tar.gz (411 kB)
        Preparing metadata (setup.py): started
        Preparing metadata (setup.py): finished with status 'done'
      Collecting mpi4py
        Using cached mpi4py-3.1.5-cp312-cp312-linux_x86_64.whl
      Collecting scikit-build-core[pyproject]
        Using cached scikit_build_core-0.6.0-py3-none-any.whl.metadata (17 kB)
      Collecting packaging>=20.9 (from scikit-build-core[pyproject])
        Using cached packaging-23.2-py3-none-any.whl.metadata (3.2 kB)
      Collecting pathspec>=0.10.1 (from scikit-build-core[pyproject])
        Using cached pathspec-0.11.2-py3-none-any.whl.metadata (19 kB)
      Collecting pyproject-metadata>=0.5 (from scikit-build-core[pyproject])
        Using cached pyproject_metadata-0.7.1-py3-none-any.whl (7.4 kB)
      Collecting numpy (from petsc4py)
        Using cached numpy-1.26.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (61 kB)
      Collecting petsc<3.21,>=3.20 (from petsc4py)
        Using cached petsc-3.20.0.tar.gz (17.0 MB)
        Preparing metadata (setup.py): started
        Preparing metadata (setup.py): finished with status 'done'
      Using cached pybind11-2.11.1-py3-none-any.whl (227 kB)
      Using cached packaging-23.2-py3-none-any.whl (53 kB)
      Using cached pathspec-0.11.2-py3-none-any.whl (29 kB)
      Using cached numpy-1.26.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (17.9 MB)
      Using cached scikit_build_core-0.6.0-py3-none-any.whl (133 kB)
      Building wheels for collected packages: petsc4py, petsc
        Building wheel for petsc4py (setup.py): started
        Building wheel for petsc4py (setup.py): finished with status 'error'
        error: subprocess-exited-with-error

        × python setup.py bdist_wheel did not run successfully.
        │ exit code: 1
        ╰─> [105 lines of output]
            running bdist_wheel
            PETSC_DIR not specified
            running build
            running build_src
            fetching build requirement 'Cython >= 3.0.0'
            using Cython 3.0.4
            cythonizing 'petsc4py/PETSc.pyx' -> 'petsc4py/PETSc.c'
            performance hint: petsc4py/PETSc/PETSc.pyx:67:5: Exception check on 'SETERR' will always require the GIL to be acquired. Declare the function as 'noexcept' if you control the definition and you're sure you don't want the function to raise exceptions.
            performance hint: petsc4py/PETSc/petscdmda.pxi:116:5: Exception check on 'DMDAGetDim' will always require the GIL to be acquired. Declare the function as 'noexcept' if you control the definition and you're sure you don't want the function to raise exceptions.
            performance hint: petsc4py/PETSc/libpetsc4py.pyx:36:5: Exception check on 'FunctionBegin' will always require the GIL to be acquired.
            Possible solutions:
                1. Declare the function as 'noexcept' if you control the definition and you're sure you don't want the function to raise exceptions.
                2. Use an 'int' return type on the function to allow an error code to be returned.
            performance hint: petsc4py/PETSc/libpetsc4py.pyx:45:5: Exception check on 'FunctionEnd' will always require the GIL to be acquired. Declare the function as 'noexcept' if you control the definition and you're sure you don't want the function to raise exceptions.
            performance hint: petsc4py/PETSc/libpetsc4py.pyx:54:5: Exception check on 'PetscSETERR' will always require the GIL to be acquired. Declare the function as 'noexcept' if you control the definition and you're sure you don't want the function to raise exceptions.
            performance hint: petsc4py/PETSc/libpetsc4py.pyx:61:5: Exception check on 'UNSUPPORTED' will always require the GIL to be acquired. Declare the function as 'noexcept' if you control the definition and you're sure you don't want the function to raise exceptions.
            performance hint: petsc4py/PETSc/libpetsc4py.pyx:72:5: Exception check on 'addRef' will always require the GIL to be acquired.
            Possible solutions:
                1. Declare the function as 'noexcept' if you control the definition and you're sure you don't want the function to raise exceptions.
                2. Use an 'int' return type on the function to allow an error code to be returned.
            performance hint: petsc4py/PETSc/libpetsc4py.pyx:76:5: Exception check on 'delRef' will always require the GIL to be acquired.
            Possible solutions:
                1. Declare the function as 'noexcept' if you control the definition and you're sure you don't want the function to raise exceptions.
                2. Use an 'int' return type on the function to allow an error code to be returned.
            performance hint: petsc4py/PETSc/PETSc.pyx:272:5: Exception check on 'traceback' will always require the GIL to be acquired. Declare the function as 'noexcept' if you control the definition and you're sure you don't want the function to raise exceptions.
            performance hint: petsc4py/PETSc/PETSc.pyx:312:5: Exception check on 'PetscPythonErrorHandler' will always require the GIL to be acquired. Declare the function as 'noexcept' if you control the definition and you're sure you don't want the function to raise exceptions.
            performance hint: petsc4py/PETSc/PETSc.pyx:79:16: Exception check will always require the GIL to be acquired. Declare the function as 'noexcept' if you control the definition and you're sure you don't want the function to raise exceptions.
            performance hint: petsc4py/PETSc/libpetsc4py.pyx:495:17: Exception check will always require the GIL to be acquired.
            Possible solutions:
                1. Declare the function as 'noexcept' if you control the definition and you're sure you don't want the function to raise exceptions.
                2. Use an 'int' return type on the function to allow an error code to be returned.
            performance hint: petsc4py/PETSc/libpetsc4py.pyx:509:22: Exception check will always require the GIL to be acquired. Declare the function as 'noexcept' if you control the definition and you're sure you don't want the function to raise exceptions.
            performance hint: petsc4py/PETSc/libpetsc4py.pyx:1335:17: Exception check will always require the GIL to be acquired.
            Possible solutions:
                1. Declare the function as 'noexcept' if you control the definition and you're sure you don't want the function to raise exceptions.
                2. Use an 'int' return type on the function to allow an error code to be returned.
            performance hint: petsc4py/PETSc/libpetsc4py.pyx:1344:22: Exception check will always require the GIL to be acquired. Declare the function as 'noexcept' if you control the definition and you're sure you don't want the function to raise exceptions.
            performance hint: petsc4py/PETSc/libpetsc4py.pyx:1396:17: Exception check will always require the GIL to be acquired.
            Possible solutions:
                1. Declare the function as 'noexcept' if you control the definition and you're sure you don't want the function to raise exceptions.
                2. Use an 'int' return type on the function to allow an error code to be returned.
            performance hint: petsc4py/PETSc/libpetsc4py.pyx:1398:22: Exception check will always require the GIL to be acquired. Declare the function as 'noexcept' if you control the definition and you're sure you don't want the function to raise exceptions.
            performance hint: petsc4py/PETSc/libpetsc4py.pyx:1641:17: Exception check will always require the GIL to be acquired.
            Possible solutions:
                1. Declare the function as 'noexcept' if you control the definition and you're sure you don't want the function to raise exceptions.
                2. Use an 'int' return type on the function to allow an error code to be returned.
            performance hint: petsc4py/PETSc/libpetsc4py.pyx:1650:22: Exception check will always require the GIL to be acquired. Declare the function as 'noexcept' if you control the definition and you're sure you don't want the function to raise exceptions.
            performance hint: petsc4py/PETSc/libpetsc4py.pyx:1692:17: Exception check will always require the GIL to be acquired.
            Possible solutions:
                1. Declare the function as 'noexcept' if you control the definition and you're sure you don't want the function to raise exceptions.
                2. Use an 'int' return type on the function to allow an error code to be returned.
            performance hint: petsc4py/PETSc/libpetsc4py.pyx:1696:22: Exception check will always require the GIL to be acquired. Declare the function as 'noexcept' if you control the definition and you're sure you don't want the function to raise exceptions.
            performance hint: petsc4py/PETSc/libpetsc4py.pyx:1992:17: Exception check will always require the GIL to be acquired.
            Possible solutions:
                1. Declare the function as 'noexcept' if you control the definition and you're sure you don't want the function to raise exceptions.
                2. Use an 'int' return type on the function to allow an error code to be returned.
            performance hint: petsc4py/PETSc/libpetsc4py.pyx:2001:22: Exception check will always require the GIL to be acquired. Declare the function as 'noexcept' if you control the definition and you're sure you don't want the function to raise exceptions.
            performance hint: petsc4py/PETSc/libpetsc4py.pyx:2045:17: Exception check will always require the GIL to be acquired.
            Possible solutions:
                1. Declare the function as 'noexcept' if you control the definition and you're sure you don't want the function to raise exceptions.
                2. Use an 'int' return type on the function to allow an error code to be returned.
            performance hint: petsc4py/PETSc/libpetsc4py.pyx:2047:22: Exception check will always require the GIL to be acquired. Declare the function as 'noexcept' if you control the definition and you're sure you don't want the function to raise exceptions.
            performance hint: petsc4py/PETSc/libpetsc4py.pyx:2343:17: Exception check will always require the GIL to be acquired.
            Possible solutions:
                1. Declare the function as 'noexcept' if you control the definition and you're sure you don't want the function to raise exceptions.
                2. Use an 'int' return type on the function to allow an error code to be returned.
            performance hint: petsc4py/PETSc/libpetsc4py.pyx:2352:22: Exception check will always require the GIL to be acquired. Declare the function as 'noexcept' if you control the definition and you're sure you don't want the function to raise exceptions.
            performance hint: petsc4py/PETSc/libpetsc4py.pyx:2407:17: Exception check will always require the GIL to be acquired.
            Possible solutions:
                1. Declare the function as 'noexcept' if you control the definition and you're sure you don't want the function to raise exceptions.
                2. Use an 'int' return type on the function to allow an error code to be returned.
            performance hint: petsc4py/PETSc/libpetsc4py.pyx:2411:22: Exception check will always require the GIL to be acquired. Declare the function as 'noexcept' if you control the definition and you're sure you don't want the function to raise exceptions.
            performance hint: petsc4py/PETSc/libpetsc4py.pyx:2759:17: Exception check will always require the GIL to be acquired.
            Possible solutions:
                1. Declare the function as 'noexcept' if you control the definition and you're sure you don't want the function to raise exceptions.
                2. Use an 'int' return type on the function to allow an error code to be returned.
            performance hint: petsc4py/PETSc/libpetsc4py.pyx:2768:22: Exception check will always require the GIL to be acquired. Declare the function as 'noexcept' if you control the definition and you're sure you don't want the function to raise exceptions.
            performance hint: petsc4py/PETSc/PETSc.pyx:324:24: Exception check will always require the GIL to be acquired. Declare the function as 'noexcept' if you control the definition and you're sure you don't want the function to raise exceptions.
            running build_py
            creating build
            creating build/lib.linux-x86_64-cpython-312
            creating build/lib.linux-x86_64-cpython-312/petsc4py
            copying src/petsc4py/PETSc.py -> build/lib.linux-x86_64-cpython-312/petsc4py
            copying src/petsc4py/__init__.py -> build/lib.linux-x86_64-cpython-312/petsc4py
            copying src/petsc4py/__main__.py -> build/lib.linux-x86_64-cpython-312/petsc4py
            copying src/petsc4py/typing.py -> build/lib.linux-x86_64-cpython-312/petsc4py
            creating build/lib.linux-x86_64-cpython-312/petsc4py/lib
            copying src/petsc4py/lib/__init__.py -> build/lib.linux-x86_64-cpython-312/petsc4py/lib
            copying src/petsc4py/PETSc.pxd -> build/lib.linux-x86_64-cpython-312/petsc4py
            copying src/petsc4py/PETSc.h -> build/lib.linux-x86_64-cpython-312/petsc4py
            copying src/petsc4py/PETSc_api.h -> build/lib.linux-x86_64-cpython-312/petsc4py
            creating build/lib.linux-x86_64-cpython-312/petsc4py/include
            creating build/lib.linux-x86_64-cpython-312/petsc4py/include/petsc4py
            copying src/petsc4py/include/petsc4py/numpy.h -> build/lib.linux-x86_64-cpython-312/petsc4py/include/petsc4py
            copying src/petsc4py/include/petsc4py/petsc4py.h -> build/lib.linux-x86_64-cpython-312/petsc4py/include/petsc4py
            copying src/petsc4py/include/petsc4py/pybuffer.h -> build/lib.linux-x86_64-cpython-312/petsc4py/include/petsc4py
            copying src/petsc4py/include/petsc4py/pyscalar.h -> build/lib.linux-x86_64-cpython-312/petsc4py/include/petsc4py
            copying src/petsc4py/include/petsc4py/petsc4py.i -> build/lib.linux-x86_64-cpython-312/petsc4py/include/petsc4py
            copying src/petsc4py/py.typed -> build/lib.linux-x86_64-cpython-312/petsc4py
            copying src/petsc4py/__init__.pyi -> build/lib.linux-x86_64-cpython-312/petsc4py
            copying src/petsc4py/__main__.pyi -> build/lib.linux-x86_64-cpython-312/petsc4py
            copying src/petsc4py/lib/__init__.pyi -> build/lib.linux-x86_64-cpython-312/petsc4py/lib
            copying src/petsc4py/lib/petsc.cfg -> build/lib.linux-x86_64-cpython-312/petsc4py/lib
            running build_ext
            error: PETSc not found
            [end of output]

        note: This error originates from a subprocess, and is likely not a problem with pip.
        ERROR: Failed building wheel for petsc4py
        Running setup.py clean for petsc4py
        Building wheel for petsc (setup.py): started
        Building wheel for petsc (setup.py): finished with status 'error'
        error: subprocess-exited-with-error

        × python setup.py bdist_wheel did not run successfully.
        │ exit code: 1
        ╰─> [1 lines of output]
            petsc: this package cannot be built as a wheel
            [end of output]

        note: This error originates from a subprocess, and is likely not a problem with pip.
        ERROR: Failed building wheel for petsc
        Running setup.py clean for petsc
      Failed to build petsc4py petsc
      ERROR: Could not build wheels for petsc4py, petsc, which is required to install pyproject.toml-based projects
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× pip subprocess to install build dependencies did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.
francesco-ballarin commented 10 months ago

Hi @hermanmakhm , as of today, with https://github.com/FEniCS/dolfinx/pull/2707 merged, to install the pybind11 wrappers of dolfinx you have to run

python3 -m pip install scikit-build-core[pyproject]
python3 -m pip install --check-build-dependencies --no-build-isolation .

(i.e., similar instructions as the one I provided you for multiphenicsx).

I now merged this to main, so you don't have to change multiphenicsx branch anymore.

The error

ERROR: Some build dependencies for file:///tmp_user/sator/hmmak/multiphenicsx conflict with the backend dependencies: fenics-dolfinx==0.7.0 is incompatible with fenics-dolfinx >=0.8.0.dev0, <0.9.0.

is expected, in the sense that now I better enforce the fact that the main branch of multiphenicsx is compatible with the main branch of dolfinx.

I close this issue since I have finished working on it, but I'll be still be very happy to read your feedback on your configuration.

francesco-ballarin commented 10 months ago

Done in https://github.com/multiphenics/multiphenicsx/commit/f78a463703d8632bfc16b310fceb6a2c479ebe06