search

hel-astro-lab / runko

Modern C++/python CPU/GPU plasma toolbox
https://runko.readthedocs.io
MIT License
47 stars 19 forks source link

Compilation #41

Closed Tissot11 closed 4 years ago

Tissot11 commented 4 years ago

I'm trying to build Runko on our cluster. During the make command I encounter few errors, which are listed below

error: use of deleted function 'fields::Binomial2<2ul>::Binomial2(int, int, int)' inline Class construct_or_initialize(Args &&...args) { return new Class{std::forward(args)...}; } ^~~~~~~~~~ error: use of deleted function 'fields::General3p<2ul>::General3p(int, int, int)' inline Class construct_or_initialize(Args &&...args) { return new Class{std::forward(args)...}; } ^~~~~~~~~~ error: no matching function for call to 'fields::Filter<2ul>::Filter()'

error: use of deleted function 'fields::General3pStrided<2ul>::General3pStrided(int, int, int)' inline Class *construct_or_initialize(Args &&...args) { return new Class{std::forward(args)...}; }

error: no matching function for call to 'fields::Filter<2ul>::Filter()'

I think I have all the requirements installed. Instead of export CC=mpicc and export CXX=mpic++, I had to build aliases for them. Please let me know how can I fix this?

natj commented 4 years ago

Seems to me like a C++ compiler problem or something to do with pybind11 version since the compilation fails trying to bind normal classes. Do you have a modern C++ compiler and up-to-date version of pybind11?

What is the output of clean cmake .. evaluation?

Tissot11 commented 4 years ago

cmake .. builds successfully. We have gcc 6.2.0 which we have been using for a while. Is it too old. I can compile later on my Mac which has gcc 9.3. I use python 3.5 and I don't how to check pybind11 version.

natj commented 4 years ago

It is most likely the gcc that is too old and does not understand some of the bindings. Oldest I have tried is gcc7 that was working.

Is the error only showing up with Filter classes? If there is something special in them I can try fixing it. However, if the error persists for all the class bindings then it most likely means you are out of luck with gcc6 and need to upgrade to some modern C++ compiler.

You can check this by running e.g. make -j8 and checking if some things compile successfully or if all of them give the same error.

Tissot11 commented 4 years ago

Till now I have only seen it with Filter , General and Binomial as I pasted some commands above. On my Mac I can't compile it now since it asks for mpic++ but I have mpicxx. Setting up aliases didn't work also. Can I modify somewhere this to mpicxx?

Update: Now I compiled with gcc 9.2.0 and openmpi 4.0 and I reinstalled mpi4py but on. compiling I get the same error as before.

natj commented 4 years ago

Ok, thanks! I looked into this and we might trigger https://github.com/natj/runko/blob/master/bindings/pyfields.c%2B%2B#L394 Basically there is a pybind hack in the bindings to make the filters have correct method bindings.

Strangely I can not reproduce this on linux with gcc7, gcc10, or with clang; with all of these the filter classes bind okay. I also tested with building against latest pybind11 master branch and it also works. Have you followed the installation steps here https://runko.readthedocs.io/en/latest/installation.html ?

If the problem does not go away by making sure that all the dependencies are correct (i.e. you are really using the latest pybind11 via runko/corgi/pybind11 and your compiler is modern (i.e. $CXX --version shows that you are really using the mpi + c++ compiler you think you are) you could temporarily hack this by commenting the filter bindings out: These are on line 387 - 421 on runko/bindings/pyfields.c++.

What comes to macos and mpich I have only tested with openmpi. Using a different mpi compiler wrapper should be possible though by making sure that export CXX=mpicxx. Alternatively you can use -D CMAKE_CXX_COMPILER=mpicxx as an extra flag when you run cmake ...

Tissot11 commented 4 years ago

On our cluster (running Scientific Linux), -CC --version_ and CXX --version show this.

gcc-9.2.0 (GCC) 9.2.0 Copyright (C) 2019 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

g++-9.2.0 (GCC) 9.2.0 Copyright (C) 2019 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Pybind11 is pybind11 v2.4.dev4 and I have used all the steps as written on the link you suggested. Finally I commented out line 387-420 but still got the error

_error: use of deleted function 'fields::Binomial2<3ul>::Binomial2(int, int, int)' inline Class *construct_orinitialize(Args &&...args) { return new Class{std::forward(args)...}; }

I only have gcc-9 and gcc-6 available on the cluster. On my mac I use Macports and I followed all the steps expect I had to create aliases since openmpi or installing py35-mpi4py that installed also mpich on my mac, but I only have mpicxx available. I tried to use gcc10 on my Mac, but it failed again saying

_/Users/naveen/CompiledCodes/runko/bindings/../vlasov/tasker.h:5:10: fatal error: 'omp.h' file not found

This I believe is a xcode problem, which is extremely annoying.

Tissot11 commented 4 years ago

After fixing xcode error I tried again, It shows error on the test stages

_test_amr (unittest.loader._FailedTest) ... ERROR test_digital_filters (unittest.loader._FailedTest) ... ERROR test_fields (unittest.loader._FailedTest) ... ERROR test_grid (unittest.loader._FailedTest) ... ERROR test_io (unittest.loader._FailedTest) ... ERROR test_pic (unittest.loader._FailedTest) ... ERROR test_rad (unittest.loader._FailedTest) ... ERROR test_tile (unittest.loader._FailedTest) ... ERROR

====================================================================== ERROR: test_amr (unittest.loader._FailedTest) ImportError: Failed to import test module: test_amr Traceback (most recent call last): File "/opt/local/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/unittest/loader.py", line 436, in _find_test_path module = self._get_module_from_name(name) File "/opt/local/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/unittest/loader.py", line 377, in _get_module_from_name import(name) File "/Users/naveen/Compiled_Codes/runko/tests/testamr.py", line 1, in from mpi4py import MPI ModuleNotFoundError: No module named 'mpi4py'

Though I have installed py35-mpi4py but it still complains about this.

natj commented 4 years ago

MacOS

Most likely there is a mixup between python versions (esp. if you use anaconda etc). You can try manually running the tests in runko main directory with:

 python3 -m unittest discover -s tests/ -v

where python3 (or whatever you use) is the python interpreter where you do have mpi4py installed. If this works it just means that your python paths are locally mixed.

MacOS does not make HPC code development easy. Even if you installed a modern compiler like gcc9 you have to make sure that MPI uses it (it defaults to AppleClang which is outdated). This is why the tutorial recommends building the openmpi by hand. On top of that, people typically have multiple python versions via anaconda, virtualenv, etc so you need to make sure that the system defaults to the python you really want to use.

Cluster

Pybind11 is pybind11 v2.4.dev4 and I have used all the steps as written on the link you suggested. Finally I commented out line 387-420 but still got the error error: use of deleted function 'fields::Binomial2<3ul>::Binomial2(int, int, int)' inline Class *construct_or_initialize(Args &&...args) { return new Class{std::forward(args)...}; }

Something is wrong here. Compilation can not refer to fields::Binomial2<3ul>... if you commented it out. Easiest thing is to do a clean build by removing build directory, re-creating it, and then inside cmake .. and make.

Tissot11 commented 4 years ago

On my MacOS I can understand the problem. I only use Macports and I had already installed and use openmpi for other codes. But if I want to install mpi4py, Macports wants to install it with mpich which is going to create problems. However, on cluster I only use openmpi and I tried to install mpi4py in my virtual environment using pip command. When cmake .. from build directory, I get this info

-- The C compiler identification is GNU 4.8.5 -- The CXX compiler identification is GNU 4.8.5 -- Detecting C compiler ABI info -- Detecting C compiler ABI info - done -- Check for working C compiler: /bin/cc - skipped -- Detecting C compile features -- Detecting C compile features - done -- Detecting CXX compiler ABI info -- Detecting CXX compiler ABI info - done -- Check for working CXX compiler: /bin/c++ - skipped -- Detecting CXX compile features -- Detecting CXX compile features - done -- Found HDF5: /home/tap/kumar/l8/Code_Reposit_l8/HDF5/lib/libhdf5.so;/usr/lib64/libz.so;/usr/lib64/libdl.so;/usr/lib64/libm.so (found version "1.10.6") -- Found OpenMP_C: -fopenmp (found version "3.1") -- Found OpenMP_CXX: -fopenmp (found version "3.1") -- Found OpenMP: TRUE (found version "3.1") -- Found PythonInterp: /home/tap/kumar/l8/Code_Reposit_l8/happiVenv/bin/python3.5 (found version "3.5.2") -- Found PythonLibs: /usr/local/Packages/Python-3.5.2/lib/libpython3.5m.a -- pybind11 v2.4.dev4 -- Found MPI_C: /nfs/us1-linux/Local/Packages/openmpi-4.0.2-gcc-9.2.0/lib/libmpi.so (found version "3.1") -- Found MPI_CXX: /nfs/us1-linux/Local/Packages/openmpi-4.0.2-gcc-9.2.0/lib/libmpi.so (found version "3.1") -- Found MPI: TRUE (found version "3.1") -- Performing Test HAS_FLTO -- Performing Test HAS_FLTO - Success -- LTO enabled -- Found Doxygen: /usr/local/bin/doxygen (found version "1.7.6.1") found components: doxygen dot -- Found Sphinx: /home/tap/kumar/l8/Code_Reposit_l8/happiVenv/bin/sphinx-build -- Configuring done -- Generating done -- Build files have been written to: /home/tap/kumar/l8/Code_Reposit_l8/runko/build while at the same time typing mpicc --version gives (likewise gcc --version)

mpicc --version gcc-9.2.0 (GCC) 9.2.0 Copyright (C) 2019 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

So something is wrong with the make setting that it finds an old compiler which I don't use. In this case after commenting lines from pyfieldsc++ I get the error

`fatal error: mpi.h: No such file or directory

include

`

I don't have any clue how to fix this.

Tissot11 commented 4 years ago

On My Mac:

echo $PYTHONPATH :/Users/naveen/Compiled_Codes/runko:/Users/naveen/Compiled_Codes/runko/lib:/Users/naveen/Compiled_Codes/runko/corgi/lib:/Users/naveen/Compiled_Codes/runko/bindings/old

But it sill complains on running python -m unittest discover -s tests/ -v

that

`test_amr (unittest.loader._FailedTest) ... ERROR test_digital_filters (unittest.loader._FailedTest) ... ERROR test_fields (unittest.loader._FailedTest) ... ERROR test_grid (unittest.loader._FailedTest) ... ERROR test_io (unittest.loader._FailedTest) ... ERROR test_pic (unittest.loader._FailedTest) ... ERROR test_rad (unittest.loader._FailedTest) ... ERROR test_tile (unittest.loader._FailedTest) ... ERROR

====================================================================== ERROR: test_amr (unittest.loader._FailedTest)

ImportError: Failed to import test module: test_amr Traceback (most recent call last): File "/opt/local/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/unittest/loader.py", line 428, in _find_test_path module = self._get_module_from_name(name) File "/opt/local/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/unittest/loader.py", line 369, in _get_module_from_name import(name) File "/Users/naveen/Compiled_Codes/runko/tests/test_amr.py", line 7, in import pyrunko.tools as pyplasma ImportError: No module named 'pyrunko'

`

and so on with other modules.

Tissot11 commented 4 years ago

I did several times different installs of hdf5 and different virtual environments on my cluster. On compiling it gives same error as described above. Running on cluster

python3 -m unittest discover -s tests/ -v

yields these error messages

`test_amr (unittest.loader._FailedTest) ... ERROR test_digital_filters (unittest.loader._FailedTest) ... ERROR test_fields (unittest.loader._FailedTest) ... ERROR test_grid (unittest.loader._FailedTest) ... ERROR test_io (unittest.loader._FailedTest) ... ERROR test_pic (unittest.loader._FailedTest) ... ERROR test_rad (unittest.loader._FailedTest) ... ERROR test_tile (unittest.loader._FailedTest) ... ERROR

====================================================================== ERROR: test_amr (unittest.loader._FailedTest)

ImportError: Failed to import test module: test_amr Traceback (most recent call last): File "/usr/local/Packages/Python-3.5.2/lib/python3.5/unittest/loader.py", line 428, in _find_test_path module = self._get_module_from_name(name) File "/usr/local/Packages/Python-3.5.2/lib/python3.5/unittest/loader.py", line 369, in _get_module_from_name import(name) File "/lfs/l8/tap/kumar/Code_Reposit_l8/runko/tests/test_amr.py", line 7, in import pyrunko.tools as pyplasma ImportError: No module named 'pyrunko'

====================================================================== ERROR: test_digital_filters (unittest.loader._FailedTest)

ImportError: Failed to import test module: test_digital_filters Traceback (most recent call last): File "/usr/local/Packages/Python-3.5.2/lib/python3.5/unittest/loader.py", line 428, in _find_test_path module = self._get_module_from_name(name) File "/usr/local/Packages/Python-3.5.2/lib/python3.5/unittest/loader.py", line 369, in _get_module_from_name import(name) File "/lfs/l8/tap/kumar/Code_Reposit_l8/runko/tests/test_digital_filters.py", line 13, in import pycorgi ImportError: No module named 'pycorgi'

====================================================================== ERROR: test_fields (unittest.loader._FailedTest)

ImportError: Failed to import test module: test_fields Traceback (most recent call last): File "/usr/local/Packages/Python-3.5.2/lib/python3.5/unittest/loader.py", line 428, in _find_test_path module = self._get_module_from_name(name) File "/usr/local/Packages/Python-3.5.2/lib/python3.5/unittest/loader.py", line 369, in _get_module_from_name import(name) File "/lfs/l8/tap/kumar/Code_Reposit_l8/runko/tests/test_fields.py", line 8, in import pycorgi ImportError: No module named 'pycorgi'

====================================================================== ERROR: test_grid (unittest.loader._FailedTest)

ImportError: Failed to import test module: test_grid Traceback (most recent call last): File "/usr/local/Packages/Python-3.5.2/lib/python3.5/unittest/loader.py", line 428, in _find_test_path module = self._get_module_from_name(name) File "/usr/local/Packages/Python-3.5.2/lib/python3.5/unittest/loader.py", line 369, in _get_module_from_name import(name) File "/lfs/l8/tap/kumar/Code_Reposit_l8/runko/tests/test_grid.py", line 5, in import pycorgi ImportError: No module named 'pycorgi'

====================================================================== ERROR: test_io (unittest.loader._FailedTest)

ImportError: Failed to import test module: test_io Traceback (most recent call last): File "/usr/local/Packages/Python-3.5.2/lib/python3.5/unittest/loader.py", line 428, in _find_test_path module = self._get_module_from_name(name) File "/usr/local/Packages/Python-3.5.2/lib/python3.5/unittest/loader.py", line 369, in _get_module_from_name import(name) File "/lfs/l8/tap/kumar/Code_Reposit_l8/runko/tests/test_io.py", line 7, in import pycorgi ImportError: No module named 'pycorgi'

====================================================================== ERROR: test_pic (unittest.loader._FailedTest)

ImportError: Failed to import test module: test_pic Traceback (most recent call last): File "/usr/local/Packages/Python-3.5.2/lib/python3.5/unittest/loader.py", line 428, in _find_test_path module = self._get_module_from_name(name) File "/usr/local/Packages/Python-3.5.2/lib/python3.5/unittest/loader.py", line 369, in _get_module_from_name import(name) File "/lfs/l8/tap/kumar/Code_Reposit_l8/runko/tests/test_pic.py", line 8, in import pytools # runko python tools File "/lfs/l8/tap/kumar/Code_Reposit_l8/runko/pytools/init.py", line 7, in from .load_grid import * File "/lfs/l8/tap/kumar/Code_Reposit_l8/runko/pytools/load_grid.py", line 5, in import pycorgi ImportError: No module named 'pycorgi'

====================================================================== ERROR: test_rad (unittest.loader._FailedTest)

ImportError: Failed to import test module: test_rad Traceback (most recent call last): File "/usr/local/Packages/Python-3.5.2/lib/python3.5/unittest/loader.py", line 428, in _find_test_path module = self._get_module_from_name(name) File "/usr/local/Packages/Python-3.5.2/lib/python3.5/unittest/loader.py", line 369, in _get_module_from_name import(name) File "/lfs/l8/tap/kumar/Code_Reposit_l8/runko/tests/test_rad.py", line 7, in import pycorgi ImportError: No module named 'pycorgi'

====================================================================== ERROR: test_tile (unittest.loader._FailedTest)

ImportError: Failed to import test module: test_tile Traceback (most recent call last): File "/usr/local/Packages/Python-3.5.2/lib/python3.5/unittest/loader.py", line 428, in _find_test_path module = self._get_module_from_name(name) File "/usr/local/Packages/Python-3.5.2/lib/python3.5/unittest/loader.py", line 369, in _get_module_from_name import(name) File "/lfs/l8/tap/kumar/Code_Reposit_l8/runko/tests/test_tile.py", line 5, in import pyrunko ImportError: No module named 'pyrunko'

Ran 8 tests in 0.001s

FAILED (errors=8) `

Tissot11 commented 4 years ago

echo $PYTHONPATH is

echo $PYTHONPATH :/home/tap/kumar/l8/Code_Reposit_l8/runko:/home/tap/kumar/l8/Code_Reposit_l8/runko/lib:/home/tap/kumar/l8/Code_Reposit_l8/runko/corgi/lib:/home/tap/kumar/l8/Code_Reposit_l8/runko/bindings/old

natj commented 4 years ago

Few things I noticed that might or might not help you:

Tissot11 commented 4 years ago

I made the changes and echo $PYTHONPATH looks this

/home/tap/kumar/l8/Code_Reposit_l8/NewVirtual/bin:/home/tap/kumar/l8/Code_Reposit_l8/runko:/home/tap/kumar/l8/Code_Reposit_l8/runko/lib:/home/tap/kumar/l8/Code_Reposit_l8/runko/corgi/lib:/home/tap/kumar/l8/Code_Reposit_l8/runko/bindings/old

However, running python3 -m unittest discover -s tests/ -v results in the same error as before on the cluster.

I have written our system adminitrator about gcc issue and I'm waiting for his answer.

Tissot11 commented 4 years ago

Could it be something wrong with cmake? I'm using cmake version 3.18.2.

Tissot11 commented 4 years ago

I could fix the gcc related issues on the cluster. The build goes fine

cmake .. -- The C compiler identification is GNU 9.2.0 -- The CXX compiler identification is GNU 9.2.0 -- Detecting C compiler ABI info -- Detecting C compiler ABI info - done -- Check for working C compiler: /usr/local/Packages/openmpi-4.0.2-gcc-9.2.0/bin/mpicc - skipped -- Detecting C compile features -- Detecting C compile features - done -- Detecting CXX compiler ABI info -- Detecting CXX compiler ABI info - done -- Check for working CXX compiler: /usr/local/Packages/openmpi-4.0.2-gcc-9.2.0/bin/mpic++ - skipped -- Detecting CXX compile features -- Detecting CXX compile features - done -- Found HDF5: /home/tap/kumar/l8/Code_Reposit_l8/NewVirtual/HDF5/lib/libhdf5.so;/usr/lib64/libz.so;/usr/lib64/libdl.so;/usr/lib64/libm.so (found version "1.10.7") -- Found OpenMP_C: -fopenmp (found version "4.5") -- Found OpenMP_CXX: -fopenmp (found version "4.5") -- Found OpenMP: TRUE (found version "4.5") -- Found PythonInterp: /home/tap/kumar/l8/Code_Reposit_l8/NewVirtual/bin/python3.5 (found version "3.5.2") -- Found PythonLibs: /usr/local/Packages/Python-3.5.2/lib/libpython3.5m.a -- pybind11 v2.4.dev4 -- Found MPI_C: /usr/local/Packages/openmpi-4.0.2-gcc-9.2.0/bin/mpicc (found version "3.1") -- Found MPI_CXX: /usr/local/Packages/openmpi-4.0.2-gcc-9.2.0/bin/mpic++ (found version "3.1") -- Found MPI: TRUE (found version "3.1") -- Performing Test HAS_FLTO -- Performing Test HAS_FLTO - Success -- LTO enabled -- Found Doxygen: /usr/local/bin/doxygen (found version "1.7.6.1") found components: doxygen dot -- Found Sphinx: /home/tap/kumar/l8/Code_Reposit_l8/NewVirtual/bin/sphinx-build -- Configuring done -- Generating done -- Build files have been written to: /home/tap/kumar/l8/Code_Reposit_l8/runko/build

However, make commands in following error which is different than before.

[ 2%] Building CXX object bindings/CMakeFiles/pyrunko.dir/runko.c++.o [ 4%] Building CXX object bindings/CMakeFiles/pyrunko.dir/pytools.c++.o [ 6%] Building CXX object bindings/CMakeFiles/pyrunko.dir//tools/hilbert.c++.o [ 8%] Building CXX object bindings/CMakeFiles/pyrunko.dir/pyfields.c++.o [ 10%] Building CXX object bindings/CMakeFiles/pyrunko.dir//em-fields/tile.c++.o [ 12%] Building CXX object bindings/CMakeFiles/pyrunko.dir//em-fields/damping_tile.c++.o [ 14%] Building CXX object bindings/CMakeFiles/pyrunko.dir//em-fields/propagator/fdtd2.c++.o [ 16%] Building CXX object bindings/CMakeFiles/pyrunko.dir//em-fields/propagator/fdtd4.c++.o [ 18%] Building CXX object bindings/CMakeFiles/pyrunko.dir//em-fields/filters/digital.c++.o [ 20%] Building CXX object bindings/CMakeFiles/pyrunko.dir/pyvlv.c++.o [ 22%] Building CXX object bindings/CMakeFiles/pyrunko.dir//vlasov/tile.c++.o [ 24%] Building CXX object bindings/CMakeFiles/pyrunko.dir//vlasov/boundaries/piston.c++.o [ 26%] Building CXX object bindings/CMakeFiles/pyrunko.dir//vlasov/boundaries/outflow.c++.o [ 28%] Building CXX object bindings/CMakeFiles/pyrunko.dir//vlasov/momentum-solvers/amr_momentum_solver.c++.o [ 30%] Building CXX object bindings/CMakeFiles/pyrunko.dir//vlasov/momentum-solvers/bwd_lagrangian.c++.o [ 32%] Building CXX object bindings/CMakeFiles/pyrunko.dir/pypic.c++.o [ 34%] Building CXX object bindings/CMakeFiles/pyrunko.dir//pic/tile.c++.o [ 36%] Building CXX object bindings/CMakeFiles/pyrunko.dir//pic/particle.c++.o [ 38%] Building CXX object bindings/CMakeFiles/pyrunko.dir//pic/boundaries/wall.c++.o [ 40%] Building CXX object bindings/CMakeFiles/pyrunko.dir//pic/boundaries/piston.c++.o [ 42%] Building CXX object bindings/CMakeFiles/pyrunko.dir//pic/pushers/boris.c++.o [ 44%] Building CXX object bindings/CMakeFiles/pyrunko.dir//pic/pushers/boris_drag.c++.o [ 46%] Building CXX object bindings/CMakeFiles/pyrunko.dir//pic/pushers/vay.c++.o [ 48%] Building CXX object bindings/CMakeFiles/pyrunko.dir//pic/interpolators/linear.c++.o [ 51%] Building CXX object bindings/CMakeFiles/pyrunko.dir//pic/depositers/zigzag.c++.o [ 53%] Building CXX object bindings/CMakeFiles/pyrunko.dir/pyrad.c++.o [ 55%] Building CXX object bindings/CMakeFiles/pyrunko.dir/pyffe.c++.o [ 57%] Building CXX object bindings/CMakeFiles/pyrunko.dir//ffe/tile.c++.o [ 59%] Building CXX object bindings/CMakeFiles/pyrunko.dir/__/ffe/skinny_yee.c++.o [ 61%] Building CXX object bindings/CMakeFiles/pyrunko.dir//ffe/currents/rffe.c++.o [ 63%] Building CXX object bindings/CMakeFiles/pyrunko.dir//io/writers/writer.c++.o [ 65%] Building CXX object bindings/CMakeFiles/pyrunko.dir//io/readers/reader.c++.o [ 67%] Building CXX object bindings/CMakeFiles/pyrunko.dir//io/snapshots/fields.c++.o [ 69%] Building CXX object bindings/CMakeFiles/pyrunko.dir//io/snapshots/test_prtcls.c++.o [ 71%] Building CXX object bindings/CMakeFiles/pyrunko.dir/__/io/snapshots/pic_moments.c++.o [ 73%] Building CXX object bindings/CMakeFiles/pyrunko.dir/__/io/snapshots/field_slices.c++.o [ 75%] Linking CXX shared module ../../lib/pyrunko.cpython-35m-x86_64-linux-gnu.so [ 75%] Built target pyrunko Scanning dependencies of target pycorgi [ 77%] Building CXX object corgi/pycorgi/CMakeFiles/pycorgi.dir/pycorgi.c++.o [ 79%] Linking CXX shared module ../../../corgi/lib/pycorgi.cpython-35m-x86_64-linux-gnu.so [ 79%] Built target pycorgi Scanning dependencies of target check-runko test_fill (test_amr.Basics) ... ok test_indexing (test_amr.Basics) ... ok test_init (test_digital_filters.FilterTests) ... ERROR test_normalization (test_digital_filters.FilterTests) ... ERROR test_normalization3D (test_digital_filters.FilterTests) ... ok test_2D_tile_memory_bug (test_fields.Communications) ... ok test_3D_mesh_memory_bug (test_fields.Communications) ... ok test_3D_tile_memory_bug (test_fields.Communications) ... ok test_exchangeCurrents3D (test_fields.Communications) ... ok test_neighs3D (test_fields.Communications) ... ok test_tile_indices2D (test_fields.Communications) ... ok test_tile_indices3D (test_fields.Communications) ... ok test_updateBoundaries (test_fields.Communications) ... ok test_updateBoundaries2D (test_fields.Communications) ... ok test_updateBoundaries3D (test_fields.Communications) ... ok test_propagators_1d (test_fields.FLD_inits) ... ok test_propagators_2d (test_fields.FLD_inits) ... ok test_node1D (test_grid.Initialization) ... ok test_node2D (test_grid.Initialization) ... ok test_write_Mesh3V (test_io.IO) ... [--] [--] ok test_write_fields1D (test_io.IO) ... ok test_write_fields2D (test_io.IO) ... ok test_write_pic2D (test_io.IO) ... ok test_communication (test_pic.PIC) ... ok test_communication3D (test_pic.PIC) ... ok test_const_field_interpolation (test_pic.PIC) ... ok test_current_array_aligning (test_pic.PIC) ... ERROR test_current_deposit (test_pic.PIC) ... ok test_current_exchange (test_pic.PIC) test that current exchange is producing hand-build array ... ok test_linear_field_interpolation (test_pic.PIC) ... ok test_test_particle_initialization (test_pic.PIC) ... ok test_blackbody (test_rad.radiation) ... Energy of photons: 0.037099356877371883 /lfs/l8/tap/kumar/Code_Reposit_l8/runko/tests/test_rad.py:253: RuntimeWarning: overflow encountered in exp bbrad_sum = np.sum(3.0e122.edges**3/(np.exp(edges/kTbb)-1.0)) Blackbody energy: 9.689469501275774e-12 ok test_initialization (test_rad.radiation) ... ok test_inheritance (test_tile.Initialization) ... ok

====================================================================== ERROR: test_init (test_digital_filters.FilterTests)

Traceback (most recent call last): File "/lfs/l8/tap/kumar/Code_Reposit_l8/runko/tests/test_digital_filters.py", line 141, in test_init flt1 = pyfields.twoD.Binomial2(conf.NxMesh, conf.NyMesh, conf.NzMesh) AttributeError: module 'pyrunko.fields.twoD' has no attribute 'Binomial2'

====================================================================== ERROR: test_normalization (test_digital_filters.FilterTests)

Traceback (most recent call last): File "/lfs/l8/tap/kumar/Code_Reposit_l8/runko/tests/test_digital_filters.py", line 165, in test_normalization flt = pyfields.twoD.Binomial2(conf.NxMesh, conf.NyMesh, conf.NzMesh) AttributeError: module 'pyrunko.fields.twoD' has no attribute 'Binomial2'

====================================================================== ERROR: test_current_array_aligning (test_pic.PIC)

Traceback (most recent call last): File "/lfs/l8/tap/kumar/Code_Reposit_l8/runko/tests/test_pic.py", line 1112, in test_current_array_aligning flt = pyrunko.fields.twoD.Binomial2(conf.NxMesh, conf.NyMesh, conf.NzMesh) AttributeError: module 'pyrunko.fields.twoD' has no attribute 'Binomial2'

Ran 34 tests in 7.778s

FAILED (errors=3) make[2]: [CMakeFiles/check-runko] Error 1 make[1]: [CMakeFiles/check-runko.dir/all] Error 2 make: *** [all] Error 2

echo $PYTHONPATH gives

/home/tap/kumar/l8/Code_Reposit_l8/NewVirtual/bin/python:/home/tap/kumar/l8/Code_Reposit_l8/runko:/home/tap/kumar/l8/Code_Reposit_l8/runko/lib:/home/tap/kumar/l8/Code_Reposit_l8/runko/corgi/lib:/home/tap/kumar/l8/Code_Reposit_l8/runko/bindings/old

On my Mac I still have old problems related to mpi4py. But if I can make it run on the cluster, it will be nice.

natj commented 4 years ago

The latest error in tests is only because you removed the filter bindings. Since other tests pass you should be good to go (expect that you can not use filters (like Binomial2 in your python driver scripts).

Whether you need filters is of course highly problem dependent. Note also that since you changed something in the cluster configuration and it worked you could try and uncomment the filter bindings back and see if they compile correctly after whatever changes you did.

With mac installation it sounds like its a problem of macports, pip, and other tools that mix up openmpi and the related mpi4py packages. There is no general solution here since everybody has their own unique developer tool setup. The method listed in the installation instructions is tested to work but of course it assumes that you dont have any other installation (like macports) interfering with its library versions.

Tissot11 commented 4 years ago

I would like to use filters sometimes since in many problems fluctuations are needed to be smoothened. I just uncommented lines 387 - 421 on runko/bindings/pyfields.c++ and I got this error on compiling

/home/tap/kumar/l8/Code_Reposit_l8/runko/bindings/pyfields.c++: In function 'void fields::bind_fields(pybind11::module&)': /home/tap/kumar/l8/Code_Reposit_l8/runko/bindings/pyfields.c++:388:48: error: expected primary-expression before 'fieldsfilter2d' 388 | py::class_< fields::Filter<2>, PyFilter<2> > fieldsfilter2d(m_2d, "Filter"); | ^~~~~~~~~~~~~~ /home/tap/kumar/l8/Code_Reposit_l8/runko/bindings/pyfields.c++:389:3: error: 'fieldsfilter2d' was not declared in this scope 389 | fieldsfilter2d | ^~~~~~~~~~~~~~ make[2]: *** [bindings/CMakeFiles/pyrunko.dir/pyfields.c++.o] Error 1 make[1]: *** [bindings/CMakeFiles/pyrunko.dir/all] Error 2 make: *** [all] Error 2

On our cluster the problem is sometimes with the Path. So I had to use following commands

PYTHON_CONFIG=/home/tap/kumar/l8/Code_Reposit_l8/NewVirtual/bin/python-config HDF5_ROOT=/home/tap/kumar/l8/Code_Reposit_l8/Packages/HDF5 LDFLAGS='-Wl,-rpath,/home/tap/kumar/l8/Code_Reposit_l8/Packages/HDF5/lib,-rpath,/home/tap/kumar/l8/Code_Reposit_l8/NewVirtual/lib' cmake ..

and

PYTHON_CONFIG=/home/tap/kumar/l8/Code_Reposit_l8/NewVirtual/bin/python-config HDF5_ROOT=/home/tap/kumar/l8/Code_Reposit_l8/Packages/HDF5 LDFLAGS='-Wl,-rpath,/home/tap/kumar/l8/Code_Reposit_l8/Packages/HDF5/lib,-rpath,/home/tap/kumar/l8/Code_Reposit_l8/NewVirtual/lib' make

Since anyway I need to run the code on the cluster, I will like to get it working there. On my Mac I can fix problems later on, maybe when Big Sur comes out. Then I can clean install everything and use your method of installation on Mac.

Tissot11 commented 4 years ago

After reinstalling hdf5 1.10.6, mpi4py 3.0.2, h5py 2.10, and uncommenting all the filters related lines in pyfields.c++ and test_pic.py, test_digital_filters.py, I could compile your code on the cluster. I'll run some simulations soon to see if everything is working fine. Strangely on my Mac, in miniconda3 environment, I could compile the code without any commenting of filters related lines. This is strange because miniconda3 doesn't even have proper parallel installations of mpi4py and hdf5, h5py.

Tissot11 commented 4 years ago

I had removed Macports from my Mac and used miniconda3 for compiling Runko.

Tissot11 commented 4 years ago

On running the shock_mini simulation, I see following error. This also happens when I compile and run a version of runko with filters related lines disabled as mentioned. Compilation goes fine without any errors. I guess the targets built are being built without errors but this building is not correct. A colleague has tried on his Linux desktop and encounter the same issues. Please look at it and let us know. I have spent a lot of time time fixing whatever issues I could fix by myself.

Running pic.py with 16 MPI processes. Initializing problem setup... initializing simulation... injected: e- prtcls: 800 e+ prtcls: 800 Traceback (most recent call last): File "pic.py", line 211, in flt = pyfld.Binomial2(conf.NxMesh, conf.NyMesh, conf.NzMesh) AttributeError: module 'pyrunko.fields.twoD' has no attribute 'Binomial2' Traceback (most recent call last): File "pic.py", line 211, in flt = pyfld.Binomial2(conf.NxMesh, conf.NyMesh, conf.NzMesh) AttributeError: module 'pyrunko.fields.twoD' has no attribute 'Binomial2' Traceback (most recent call last): File "pic.py", line 211, in flt = pyfld.Binomial2(conf.NxMesh, conf.NyMesh, conf.NzMesh) AttributeError: module 'pyrunko.fields.twoD' has no attribute 'Binomial2' Traceback (most recent call last): File "pic.py", line 211, in flt = pyfld.Binomial2(conf.NxMesh, conf.NyMesh, conf.NzMesh) AttributeError: module 'pyrunko.fields.twoD' has no attribute 'Binomial2' Traceback (most recent call last): File "pic.py", line 211, in flt = pyfld.Binomial2(conf.NxMesh, conf.NyMesh, conf.NzMesh) AttributeError: module 'pyrunko.fields.twoD' has no attribute 'Binomial2' Traceback (most recent call last): File "pic.py", line 211, in flt = pyfld.Binomial2(conf.NxMesh, conf.NyMesh, conf.NzMesh) AttributeError: module 'pyrunko.fields.twoD' has no attribute 'Binomial2' Traceback (most recent call last): File "pic.py", line 211, in flt = pyfld.Binomial2(conf.NxMesh, conf.NyMesh, conf.NzMesh) AttributeError: module 'pyrunko.fields.twoD' has no attribute 'Binomial2' Traceback (most recent call last): File "pic.py", line 211, in flt = pyfld.Binomial2(conf.NxMesh, conf.NyMesh, conf.NzMesh) AttributeError: module 'pyrunko.fields.twoD' has no attribute 'Binomial2' Traceback (most recent call last): File "pic.py", line 211, in flt = pyfld.Binomial2(conf.NxMesh, conf.NyMesh, conf.NzMesh) AttributeError: module 'pyrunko.fields.twoD' has no attribute 'Binomial2' Traceback (most recent call last): File "pic.py", line 211, in flt = pyfld.Binomial2(conf.NxMesh, conf.NyMesh, conf.NzMesh) AttributeError: module 'pyrunko.fields.twoD' has no attribute 'Binomial2' Traceback (most recent call last): File "pic.py", line 211, in flt = pyfld.Binomial2(conf.NxMesh, conf.NyMesh, conf.NzMesh) AttributeError: module 'pyrunko.fields.twoD' has no attribute 'Binomial2' Traceback (most recent call last): File "pic.py", line 211, in flt = pyfld.Binomial2(conf.NxMesh, conf.NyMesh, conf.NzMesh) AttributeError: module 'pyrunko.fields.twoD' has no attribute 'Binomial2' Traceback (most recent call last): File "pic.py", line 211, in flt = pyfld.Binomial2(conf.NxMesh, conf.NyMesh, conf.NzMesh) AttributeError: module 'pyrunko.fields.twoD' has no attribute 'Binomial2' Traceback (most recent call last): File "pic.py", line 211, in flt = pyfld.Binomial2(conf.NxMesh, conf.NyMesh, conf.NzMesh) AttributeError: module 'pyrunko.fields.twoD' has no attribute 'Binomial2' Traceback (most recent call last): File "pic.py", line 211, in flt = pyfld.Binomial2(conf.NxMesh, conf.NyMesh, conf.NzMesh) AttributeError: module 'pyrunko.fields.twoD' has no attribute 'Binomial2' Traceback (most recent call last): File "pic.py", line 211, in flt = pyfld.Binomial2(conf.NxMesh, conf.NyMesh, conf.NzMesh) AttributeError: module 'pyrunko.fields.twoD' has no attribute 'Binomial2'

Primary job terminated normally, but 1 process returned a non-zero exit code. Per user-direction, the job has been aborted.

mpirun detected that one or more processes exited with non-zero status, thus causing the job to be terminated. The first process to do so was:

Process name: [[1031,1],0] Exit code: 1

Tissot11 commented 4 years ago

On my Mac, building in miniconda3 environment as mentioned before, compilation goes fine without any errors. However, on running the same shock_mini.ini file results in different error this time

mpirun -n 4 python3 pic.py --conf shock_mini.ini

Running pic.py with 4 MPI processes. Initializing problem setup... initializing simulation... injected: e- prtcls: 3200 e+ prtcls: 3200 n_particles: 12800 stride: 1 my_n_prtcls: 3200 min_n_prtcls: 3200 cutoff_id: 3200 n_test_particles: 12800 Traceback (most recent call last): File "pic.py", line 257, in slice_xy_writer = pyfld.FieldSliceWriter( conf.outdir, AttributeError: module 'pyrunko.fields.twoD' has no attribute 'FieldSliceWriter' Traceback (most recent call last): File "pic.py", line 257, in slice_xy_writer = pyfld.FieldSliceWriter( conf.outdir, AttributeError: module 'pyrunko.fields.twoD' has no attribute 'FieldSliceWriter' Traceback (most recent call last): File "pic.py", line 257, in slice_xy_writer = pyfld.FieldSliceWriter( conf.outdir, AttributeError: module 'pyrunko.fields.twoD' has no attribute 'FieldSliceWriter' Traceback (most recent call last): File "pic.py", line 257, in slice_xy_writer = pyfld.FieldSliceWriter( conf.outdir, AttributeError: module 'pyrunko.fields.twoD' has no attribute 'FieldSliceWriter'

Primary job terminated normally, but 1 process returned a non-zero exit code. Per user-direction, the job has been aborted.

mpirun detected that one or more processes exited with non-zero status, thus causing the job to be terminated. The first process to do so was:

Process name: [[45433,1],3] Exit code: 1

Tissot11 commented 4 years ago

I forget to mention before, that probably in the list of requirements you should also add cython. Installing it could make the compilation of the code goes fine with filters related lines being present.

natj commented 4 years ago

The filter error (no Binomial2) seems strange. Are you sure there is no dublicate installation of the code (one that you compiled without filters and one where they are fixed back)? None of the tests (ran automatically for every commit) have detected this error so it seems likely that it is due to a dirty installation. Make sure to clean the build directory completely and do the cmake .. & make -j4 from scratch after every update. And check that the export path (e.g. echo $RUNKO is pointing to the correct directory).

For the FieldSliceWriter you can just comment it out in the pic.py. The saving of slices is only supported in 3D simulations. Unfortunately most of the python drivers in runko/projects are outdated since we use internal versions of different drivers tailored for each physics problem. This bug with FieldSliceWriter is of course something I can fix to the upstream too. Sorry about that & thanks for reporting it!

cython is not explicitly used anywhere so this seems strange. I'll need to investigate if pybind11 needs it since it is definitely not needed in any clean standard installation that I am testing (linux with gcc7/10/clang and macos gcc10 via homebrew).

Tissot11 commented 4 years ago

Thanks for your prompt reply and the suggestions! Indeed there was a Path problem on the cluster. After fixing it and after commenting lines from 256-265 and 583-586 in pic.py, I could run the shock.mini both on my Mac and on the cluster :). I'll run remaining tests soon.

Could you please push the new version of the code to GitHub? I really like the way you have conceptualised and written runko and I'm keen to extensively use it for my research! Also I will be interested in future to contribute to it.

natj commented 4 years ago

Closed all the pending pull requests so everything is up-to-date now. Also fixed the projects/pic-turbulence, projects/pic-shocks, and projects/ffe-packets tutorials so that they should finish without any errors.

You might want to git pull and recompile to get the latest changes.

Since the compilation works now I'll close this issue. If something else is broken, feel free to open up a new issue. For more science-related questions better contact me directly via email.