search

thliebig / openEMS

openEMS is a free and open-source electromagnetic field solver using the EC-FDTD method.
http://openEMS.de
GNU General Public License v3.0
413 stars 146 forks source link

Traceback from cython.numeric; missing package dependencies #127

Closed oberstet closed 8 months ago

oberstet commented 9 months ago

Hi there, first of all, thanks for creating openEMS and making it available as OSS! Awesome=)

Now I do have a problem with getting everything set up from source (Ubuntu 22.04, Python 3.11 from sources), Cython refuses to spill out code with:

(cpy311_7) oberstet@intel-nuci7:~/scm/3rdparty/openEMS-Project/openEMS/python$ git remote -v
origin  https://github.com/thliebig/openEMS.git (fetch)
origin  https://github.com/thliebig/openEMS.git (push)
(cpy311_7) oberstet@intel-nuci7:~/scm/3rdparty/openEMS-Project/openEMS/python$ git log -n1
commit 5f36e7f3a2367123f00999491a069aed50c6f244 (HEAD -> master, tag: v0.0.36, origin/master, origin/HEAD)
Author: Thorsten Liebig <Thorsten.Liebig@gmx.de>
Date:   Sun Oct 22 16:25:51 2023 +0200

    version 0.0.36

    Signed-off-by: Thorsten Liebig <Thorsten.Liebig@gmx.de>
(cpy311_7) oberstet@intel-nuci7:~/scm/3rdparty/openEMS-Project/openEMS/python$ python -V
Python 3.11.1
(cpy311_7) oberstet@intel-nuci7:~/scm/3rdparty/openEMS-Project/openEMS/python$ which python
/home/oberstet/cpy311_7/bin/python
(cpy311_7) oberstet@intel-nuci7:~/scm/3rdparty/openEMS-Project/openEMS/python$ pip install .
DEPRECATION: Loading egg at /home/oberstet/cpy311_7/lib/python3.11/site-packages/CSXCAD-0.6.2-py3.11-linux-x86_64.egg is deprecated. pip 24.3 will enforce this behaviour change. A possible replacement is to use pip for package installation.. Discussion can be found at https://github.com/pypa/pip/issues/12330
Processing /home/oberstet/scm/3rdparty/openEMS-Project/openEMS/python
  Preparing metadata (setup.py) ... error
  error: subprocess-exited-with-error

  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> [29 lines of output]
      warning: openEMS/_nf2ff.pxd:27:139: The keyword 'nogil' should appear at the end of the function signature line. Placing it before 'except' or 'noexcept' will be disallowed in a future version of Cython.

      Error compiling Cython file:
      ------------------------------------------------------------
      ...

      from libcpp.string cimport string
      from libcpp.vector cimport vector
      from libcpp.complex cimport complex
      from libcpp cimport bool
      cimport cython.numeric
              ^
      ------------------------------------------------------------

      openEMS/_nf2ff.pxd:23:8: 'cython.numeric' is not a valid cython.* module. Instead, use 'import cython' and then 'cython.numeric'.
      Traceback (most recent call last):
        File "<string>", line 2, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "/home/oberstet/scm/3rdparty/openEMS-Project/openEMS/python/setup.py", line 46, in <module>
          ext_modules = cythonize(extensions, language_level = "3")
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/home/oberstet/cpy311_7/lib/python3.11/site-packages/Cython/Build/Dependencies.py", line 1154, in cythonize
          cythonize_one(*args)
        File "/home/oberstet/cpy311_7/lib/python3.11/site-packages/Cython/Build/Dependencies.py", line 1321, in cythonize_one
          raise CompileError(None, pyx_file)
      Cython.Compiler.Errors.CompileError: /home/oberstet/scm/3rdparty/openEMS-Project/openEMS/python/openEMS/_nf2ff.pyx
      Compiling /home/oberstet/scm/3rdparty/openEMS-Project/openEMS/python/openEMS/_nf2ff.pyx because it changed.
      Compiling /home/oberstet/scm/3rdparty/openEMS-Project/openEMS/python/openEMS/openEMS.pyx because it changed.
      [1/2] Cythonizing /home/oberstet/scm/3rdparty/openEMS-Project/openEMS/python/openEMS/_nf2ff.pyx
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.
(cpy311_7) oberstet@intel-nuci7:~/scm/3rdparty/openEMS-Project/openEMS/python$

I've closely followed the docs https://docs.openems.de/install/manual.html#install-python-bindings-optional, and while the other parts did build, the Python bindings not.

The Python package does not restrict the Python version required (https://github.com/thliebig/openEMS/blob/master/python/setup.py), and so I assume Python 3.11 should work?

Any hints or tips would be highly appreciated! Cheers, /Tobias

oberstet commented 9 months ago

Turns out, the fix is easy:

(cpy311_7) oberstet@intel-nuci7:~/scm/3rdparty/openEMS-Project/openEMS/python$ git diff
diff --git a/python/openEMS/_nf2ff.pxd b/python/openEMS/_nf2ff.pxd
index 859f062..6cb11b5 100644
--- a/python/openEMS/_nf2ff.pxd
+++ b/python/openEMS/_nf2ff.pxd
@@ -20,7 +20,8 @@ from libcpp.string cimport string
 from libcpp.vector cimport vector
 from libcpp.complex cimport complex
 from libcpp cimport bool
-cimport cython.numeric
+
+import cython

 cdef extern from "openEMS/nf2ff.h":
     cdef cppclass cpp_nf2ff "nf2ff":

With that, I could successfully test and run Tutorials/Helical_Antenna.py

oberstet commented 8 months ago

There is another glitch (found it as I'm testing with a completely fresh, from sources build, including cpython):


(cpy312_1) oberstet@intel-nuci7:~/scm/typedefint/rfminer$ python -c "import openEMS"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/oberstet/cpy312_1/lib/python3.12/site-packages/openEMS/__init__.py", line 9, in <module>
    from openEMS.openEMS import openEMS
  File "openEMS/openEMS.pyx", line 22, in init openEMS.openEMS
  File "/home/oberstet/cpy312_1/lib/python3.12/site-packages/openEMS/nf2ff.py", line 22, in <module>
    import h5py
ModuleNotFoundError: No module named 'h5py'

after fixing https://github.com/thliebig/openEMS/pull/132/commits/190330336f16573c4ebcd78fe5ce4e4e1fe4de5c this now works

(cpy312_1) oberstet@intel-nuci7:~/scm/3rdparty/openEMS$ python -V
pip show openems
python -c "from CSXCAD.CSXCAD import ContinuousStructure"
python -c "import openEMS"
Python 3.12.1
Name: openEMS
Version: 0.0.36
Summary: Python interface for the openEMS FDTD library
Home-page: https://openEMS.de
Author: Thorsten Liebig
Author-email: Thorsten.Liebig@gmx.de
License: 
Location: /home/oberstet/cpy312_1/lib/python3.12/site-packages
Requires: h5py, numpy
Required-by:
0xCoto commented 8 months ago

I also stumbled upon the Cython import issue recently while building with VTK 9 instead of VTK 7, though I'd have to check whether that's the true cause. Do you recall whether you apt installed libvtk7-dev/libvtk7-qt-dev or libvtk9-dev/libvtk9-qt-dev? I'd have to test, although I'm not sure why that would cause such issues. I'm not aware of any recent commits that might be responsible for this.

Also, the file doesn't seem to require the import at all, so I simply got rid of the line entirely while fixing this and it built fine:

-cimport cython.numeric
oberstet commented 8 months ago

Do you recall whether you apt installed libvtk7-dev/libvtk7-qt-dev or libvtk9-dev/libvtk9-qt-dev?

I was using the standard distro VTK on Ubuntu 22.04, which is VTK 9.1

(.venv) oberstet@intel-nuci7:~/scm/typedefint/rfminer$ apt search vtk | grep 9.1

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

libvtk9-dev/jammy,now 9.1.0+really9.1.0+dfsg2-3build1 amd64  [installiert]
libvtk9-java/jammy,now 9.1.0+really9.1.0+dfsg2-3build1 amd64  [Installiert,automatisch]
libvtk9-qt-dev/jammy,now 9.1.0+really9.1.0+dfsg2-3build1 amd64  [installiert]
libvtk9.1/jammy,now 9.1.0+really9.1.0+dfsg2-3build1 amd64  [Installiert,automatisch]
libvtk9.1-qt/jammy,now 9.1.0+really9.1.0+dfsg2-3build1 amd64  [Installiert,automatisch]
python3-vtk9/jammy,now 9.1.0+really9.1.0+dfsg2-3build1 amd64  [Installiert,automatisch]
vtk9/jammy,now 9.1.0+really9.1.0+dfsg2-3build1 amd64  [Installiert,automatisch]
vtk9-doc/jammy,jammy 9.1.0+really9.1.0+dfsg2-3build1 all
vtk9-examples/jammy,jammy 9.1.0+really9.1.0+dfsg2-3build1 all
(.venv) oberstet@intel-nuci7:~/scm/typedefint/rfminer$

all of the build works using that

however https://github.com/thliebig/openEMS-Project/discussions/137

thus, I am now running VTK 9.3

(.venv) oberstet@intel-nuci7:~/scm/3rdparty/openEMS-Project$ openEMS
 ---------------------------------------------------------------------- 
 | openEMS 64bit -- version v0.0.36
 | (C) 2010-2023 Thorsten Liebig <thorsten.liebig@gmx.de>  GPL license
 ---------------------------------------------------------------------- 
    Used external libraries:
        CSXCAD -- Version: v0.6.3
        hdf5   -- Version: 1.10.7
                  compiled against: HDF5 library version: 1.10.7
        tinyxml -- compiled against: 2.6.2
        fparser
        boost  -- compiled against: 1_74
        vtk -- Version: 9.3.0
               compiled against: 9.3.0

 Usage: openEMS <FDTD_XML_FILE> [<options>...]

 <options>
    --disable-dumps     Disable all field dumps for faster simulation
    --debug-material    Dump material distribution to a vtk file for debugging
    --debug-PEC     Dump metal distribution to a vtk file for debugging
    --debug-operator    Dump operator to vtk file for debugging
    --debug-boxes       Dump e.g. probe boxes to vtk file for debugging
    --debug-CSX     Write CSX geometry file to debugCSX.xml
    --engine=<type>     Choose engine type
        --engine=fastest        fastest available engine (default)
        --engine=basic          basic FDTD engine
        --engine=sse            engine using sse vector extensions
        --engine=sse-compressed     engine using compressed operator + sse vector extensions
        --engine=multithreaded      engine using compressed operator + sse vector extensions + multithreading
    --numThreads=<n>    Force use n threads for multithreaded engine (needs: --engine=multithreaded)
    --no-simulation     only run preprocessing; do not simulate
    --dump-statistics   dump simulation statistics to 'openEMS_run_stats.txt' and 'openEMS_stats.txt'

     Additional global arguments 
    --showProbeDiscretization   Show probe discretization information
    --nativeFieldDumps      Dump all fields using the native field components
    -v,-vv,-vvv         Set debug level: 1 to 3

and the log noise is gone=) it was "harmless" functionally, but a pain on my nerves. I had to use the mouse to scroll through tons of screens all the time just to find my log messages.

Also, the file doesn't seem to require the import at all, so I simply got rid of the line entirely while fixing this and it built fine:

ah, right;) that's even better.

done: https://github.com/thliebig/openEMS/pull/132/commits/2b5b89ad10302f10b8462b04f578166d1d5ec72a