search

marcelotduarte / cx_Freeze

cx_Freeze creates standalone executables from Python scripts, with the same performance, is cross-platform and should work on any platform that Python itself works on.
https://marcelotduarte.github.io/cx_Freeze/
Other
1.36k stars 219 forks source link

ImportError: libopenblasp (cx_Freeze on linux) #653

Closed ksaurabh-cadence closed 4 years ago

ksaurabh-cadence commented 4 years ago

I am using cx-Freeze==6.1 and I still see the same issue as https://github.com/anthony-tuininga/cx_Freeze/issues/406

ImportError: libopenblasp-r0-34a18dc3.3.7.so: cannot open shared object file: No such file or directory

I am using Python 3.6.1 on RHEL 7. The missing libraries were present in numpy.libs folder at the time of packaging and looking at the cx_freeze log I see following message

copying xxx/lib/python3.6/site-packages/numpy/core/../../numpy.libs/libopenblasp-r0-34a18dc3.3.7.so -> app2/exe.linux-x86_64-3.6/lib/numpy/core/libopenblasp-r0-34a18dc3.3.7.so

While manually copying the so file into app2/exe.linux-x86_64-3.6/lib solves the issue, but I am not sure why it's copying it into numpy/core folder. Is there a way to make it copy it into app2/exe.linux-x86_64-3.6/lib if that's where the app is finally going to look for these so files?

marcelotduarte commented 4 years ago

cx_Freeze 6.2 has just been released. I see an issue with numpy 1.18.3 to 1.19 and pillow 7.x, and I'm investigating. For now, it has a workaround. pip install "numpy<1.18.3" "pillow<7"

marcelotduarte commented 4 years ago

@ksaurabh-cadence If you have a C compiler, you can test the development version: pip install -U git+https://github.com/marcelotduarte/cx_Freeze.git@develop

marcelotduarte commented 4 years ago

cx_Freeze 6.3 has just been released. Closing due to inactivity. If you had issues please re-open.

Ghostkeeper commented 3 years ago

I am running into this issue as well, using cx_Freeze 6.5.3. This is the whole error I'm getting:

Traceback (most recent call last):
  File "/build/env/python39_qt515/inst/lib/python3.8/site-packages/numpy/core/__init__.py", line 22, in <module>
  File "/build/env/python39_qt515/inst/lib/python3.8/site-packages/numpy/core/multiarray.py", line 12, in <module>
  File "/build/env/python39_qt515/inst/lib/python3.8/site-packages/numpy/core/overrides.py", line 7, in <module>
ImportError: libopenblasp-r0-5bebc122.3.13.dev.so: cannot open shared object file: No such file or directory

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/build/env/python38_qt515/inst/lib/python3.8/site-packages/cx_Freeze/initscripts/__startup__.py", line 74, in run
  File "/build/env/python38_qt515/inst/lib/python3.8/site-packages/cx_Freeze/initscripts/Console.py", line 36, in run
  File "/build/python38_qt515/build/inst/bin/cura", line 32, in <module>
  File "/build/python38_qt515/build/inst/lib/python3.8/site-packages/cura/CrashHandler.py", line 27, in <module>
  File "/build/python38_qt515/build/inst/lib/python3.8/site-packages/UM/Application.py", line 9, in <module>
  File "/build/python38_qt515/build/inst/lib/python3.8/site-packages/UM/Controller.py", line 3, in <module>
  File "/build/python38_qt515/build/inst/lib/python3.8/site-packages/UM/Scene/Iterator/DepthFirstIterator.py", line 3, in <module>
  File "/build/python38_qt515/build/inst/lib/python3.8/site-packages/UM/Scene/SceneNode.py", line 7, in <module>
  File "/build/env/python38_qt515/inst/lib/python3.8/site-packages/numpy/__init__.py", line 145, in <module>
  File "/build/env/python38_qt515/inst/lib/python3.8/site-packages/numpy/core/__init__.py", line 48, in <module>
ImportError: 

IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!

Importing the numpy C-extensions failed. This error can happen for
many reasons, often due to issues with your setup or how NumPy was
installed.

We have compiled some common reasons and troubleshooting tips at:

    https://numpy.org/devdocs/user/troubleshooting-importerror.html

Please note and check the following:

  * The Python version is: Python3.8 from "/tmp/.mount_Ultima7oE1Th/usr/bin/cura"
  * The NumPy version is: "1.20.1"

and make sure that they are the versions you expect.
Please carefully study the documentation linked above for further help.

Original error was: libopenblasp-r0-5bebc122.3.13.dev.so: cannot open shared object file: No such file or directory

This is the snippet from my build log related to packaging Numpy:

Copying data from package numpy...
copying /build/env/python38_qt515/inst/lib/python3.8/site-packages/numpy/core/_multiarray_tests.cpython-38-x86_64-linux-gnu.so -> package/usr/bin/lib/numpy/core/_multiarray_tests.cpython-38-x86_64-linux-gnu.so
copying /build/env/python38_qt515/inst/lib/python3.8/site-packages/numpy/core/_multiarray_umath.cpython-38-x86_64-linux-gnu.so -> package/usr/bin/lib/numpy/core/_multiarray_umath.cpython-38-x86_64-linux-gnu.so
creating directory package/usr/bin/lib/numpy/core/../../numpy.libs
copying /build/env/python38_qt515/inst/lib/python3.8/site-packages/numpy/core/../../numpy.libs/libopenblasp-r0-5bebc122.3.13.dev.so -> package/usr/bin/lib/numpy/core/../../numpy.libs/libopenblasp-r0-5bebc122.3.13.dev.so
WARNING: cannot find libgfortran-2e0d59d6.so.5.0.0
copying /build/env/python38_qt515/inst/lib/python3.8/site-packages/numpy/core/../../numpy.libs/libgfortran-2e0d59d6.so.5.0.0 -> package/usr/bin/lib/numpy/core/../../numpy.libs/libgfortran-2e0d59d6.so.5.0.0
WARNING: cannot find libquadmath-2d0c479f.so.0.0.0
WARNING: cannot find libz-eb09ad1d.so.1.2.3
copying /build/env/python38_qt515/inst/lib/python3.8/site-packages/numpy/core/../../numpy.libs/libquadmath-2d0c479f.so.0.0.0 -> package/usr/bin/lib/numpy/core/../../numpy.libs/libquadmath-2d0c479f.so.0.0.0
copying /build/env/python38_qt515/inst/lib/python3.8/site-packages/numpy/core/../../numpy.libs/libz-eb09ad1d.so.1.2.3 -> package/usr/bin/lib/numpy/core/../../numpy.libs/libz-eb09ad1d.so.1.2.3
copying /build/env/python38_qt515/inst/lib/python3.8/site-packages/numpy/core/_operand_flag_tests.cpython-38-x86_64-linux-gnu.so -> package/usr/bin/lib/numpy/core/_operand_flag_tests.cpython-38-x86_64-linux-gnu.so
copying /build/env/python38_qt515/inst/lib/python3.8/site-packages/numpy/core/_rational_tests.cpython-38-x86_64-linux-gnu.so -> package/usr/bin/lib/numpy/core/_rational_tests.cpython-38-x86_64-linux-gnu.so
copying /build/env/python38_qt515/inst/lib/python3.8/site-packages/numpy/core/_simd.cpython-38-x86_64-linux-gnu.so -> package/usr/bin/lib/numpy/core/_simd.cpython-38-x86_64-linux-gnu.so
copying /build/env/python38_qt515/inst/lib/python3.8/site-packages/numpy/core/_struct_ufunc_tests.cpython-38-x86_64-linux-gnu.so -> package/usr/bin/lib/numpy/core/_struct_ufunc_tests.cpython-38-x86_64-linux-gnu.so
copying /build/env/python38_qt515/inst/lib/python3.8/site-packages/numpy/core/_umath_tests.cpython-38-x86_64-linux-gnu.so -> package/usr/bin/lib/numpy/core/_umath_tests.cpython-38-x86_64-linux-gnu.so
copying /build/env/python38_qt515/inst/lib/python3.8/site-packages/numpy/fft/_pocketfft_internal.cpython-38-x86_64-linux-gnu.so -> package/usr/bin/lib/numpy/fft/_pocketfft_internal.cpython-38-x86_64-linux-gnu.so
copying /build/env/python38_qt515/inst/lib/python3.8/site-packages/numpy/linalg/_umath_linalg.cpython-38-x86_64-linux-gnu.so -> package/usr/bin/lib/numpy/linalg/_umath_linalg.cpython-38-x86_64-linux-gnu.so
copying /build/env/python38_qt515/inst/lib/python3.8/site-packages/numpy/linalg/lapack_lite.cpython-38-x86_64-linux-gnu.so -> package/usr/bin/lib/numpy/linalg/lapack_lite.cpython-38-x86_64-linux-gnu.so
copying /build/env/python38_qt515/inst/lib/python3.8/site-packages/numpy/random/_bounded_integers.cpython-38-x86_64-linux-gnu.so -> package/usr/bin/lib/numpy/random/_bounded_integers.cpython-38-x86_64-linux-gnu.so
copying /build/env/python38_qt515/inst/lib/python3.8/site-packages/numpy/random/_common.cpython-38-x86_64-linux-gnu.so -> package/usr/bin/lib/numpy/random/_common.cpython-38-x86_64-linux-gnu.so
copying /build/env/python38_qt515/inst/lib/python3.8/site-packages/numpy/random/_generator.cpython-38-x86_64-linux-gnu.so -> package/usr/bin/lib/numpy/random/_generator.cpython-38-x86_64-linux-gnu.so
copying /build/env/python38_qt515/inst/lib/python3.8/site-packages/numpy/random/_mt19937.cpython-38-x86_64-linux-gnu.so -> package/usr/bin/lib/numpy/random/_mt19937.cpython-38-x86_64-linux-gnu.so
copying /build/env/python38_qt515/inst/lib/python3.8/site-packages/numpy/random/_pcg64.cpython-38-x86_64-linux-gnu.so -> package/usr/bin/lib/numpy/random/_pcg64.cpython-38-x86_64-linux-gnu.so
copying /build/env/python38_qt515/inst/lib/python3.8/site-packages/numpy/random/_philox.cpython-38-x86_64-linux-gnu.so -> package/usr/bin/lib/numpy/random/_philox.cpython-38-x86_64-linux-gnu.so
copying /build/env/python38_qt515/inst/lib/python3.8/site-packages/numpy/random/_sfc64.cpython-38-x86_64-linux-gnu.so -> package/usr/bin/lib/numpy/random/_sfc64.cpython-38-x86_64-linux-gnu.so
copying /build/env/python38_qt515/inst/lib/python3.8/site-packages/numpy/random/bit_generator.cpython-38-x86_64-linux-gnu.so -> package/usr/bin/lib/numpy/random/bit_generator.cpython-38-x86_64-linux-gnu.so
copying /build/env/python38_qt515/inst/lib/python3.8/site-packages/numpy/random/mtrand.cpython-38-x86_64-linux-gnu.so -> package/usr/bin/lib/numpy/random/mtrand.cpython-38-x86_64-linux-gnu.so

This numpy version (1.20.1) was taken from Pip.

I'll try creating a workaround. But I would still consider this to be an issue in version 6.5.3.

marcelotduarte commented 3 years ago

@Ghostkeeper There are others open issues about this. A workaround is: https://github.com/marcelotduarte/cx_Freeze/issues/653#issuecomment-657127554

With a series of patches, mainly #964, #965, #966, #967, #968, packages distributed as manylinux wheels, like Pillow and NumPy, which use extension modules, can locate its external shared libraries. With the help of patchelf, we automatically modify the appropriate RPATH entries such that these libraries will be picked up at runtime. Please note that patchelf and the C compiler must be installed. To install patchelf in debian/ubuntu: sudo apt install patchelf Now you can test the development version: pip install -U git+https://github.com/marcelotduarte/cx_Freeze.git@main

If your project builds successfully, please notify me with a comment.

Ghostkeeper commented 3 years ago

In the end we went with the workaround similar to what ksaurabh-cadence did. It's not the nicest solution, but we've got it to work at least.

We'd prefer to not work with unstable versions of our dependencies, since it's in a project quite a few people depend on. But I'm glad to hear you're working on it (or may even already have a fix in the main branch). Thanks for your maintenance!

marcelotduarte commented 3 years ago

cx_Freeze 6.6 has just been released.