vincefn / pyvkfft

Python interface to VkFFT
MIT License
51 stars 6 forks source link

Shared object file issues on import #6

Closed kkotyk closed 3 years ago

kkotyk commented 3 years ago

When I try to import pyvkfft.cuda I get the following traceback. Any ideas on how to fix this? Thanks.

Traceback (most recent call last): File "filtering_test.py", line 9, in import pyvkfft.cuda as vk File "/laptop/dspenv/lib64/python3.6/site-packages/pyvkfft/cuda.py", line 46, in _vkfft_cuda = load_library("_vkfft_cuda") File "/laptop/dspenv/lib64/python3.6/site-packages/pyvkfft/cuda.py", line 43, in load_library return ctypes.cdll.LoadLibrary(os.path.join(os.path.dirname(file) or os.path.curdir, basename + ext)) File "/usr/lib64/python3.6/ctypes/init.py", line 426, in LoadLibrary return self._dlltype(name) File "/usr/lib64/python3.6/ctypes/init.py", line 348, in init self._handle = _dlopen(self._name, mode) OSError: /laptop/dspenv/lib64/python3.6/site-packages/pyvkfft/_vkfft_cuda.cpython-36m-x86_64-linux-gnu.so: cannot open shared object file: No such file or directory

vincefn commented 3 years ago

Without more information about 1) your system 2) how you installed pyvkfft it will be difficult to help you. Also 3) you are executing a python file but are not supplying it...

In this case most importantly you may have a cuda issue - are the cuda tools properly installed ? Does pycuda work ? Were there messages during install ?

kkotyk commented 3 years ago

I installed using pip3 install pyvkfft. The install completed with no issues. I havent tested pycuda, but it imports. I frequently use pytorch and cupy without issue though. I have the same error when I clone your code and try to run the benchmark.py file. This is on a linux system and I've installed into a virtualenv. Is there any more information I can give?

Traceback (most recent call last): File "benchmark.py", line 28, in import pyvkfft.cuda File "/laptop/dspenv/lib64/python3.6/site-packages/pyvkfft/cuda.py", line 46, in _vkfft_cuda = load_library("_vkfft_cuda") File "/laptop/dspenv/lib64/python3.6/site-packages/pyvkfft/cuda.py", line 43, in load_library return ctypes.cdll.LoadLibrary(os.path.join(os.path.dirname(file) or os.path.curdir, basename + ext)) File "/usr/lib64/python3.6/ctypes/init.py", line 426, in LoadLibrary return self._dlltype(name) File "/usr/lib64/python3.6/ctypes/init.py", line 348, in init self._handle = _dlopen(self._name, mode) OSError: /laptop/dspenv/lib64/python3.6/site-packages/pyvkfft/_vkfft_cuda.cpython-36m-x86_64-linux-gnu.so: cannot open shared object file: No such file or directory

vincefn commented 3 years ago

Does import pyvkfft.opencl work ?

To make a reproducible test, you should test in a fresh virtual environment (you can have weird results if you mix pip install and python setup.py install).

In that fresh environment, install using pip install pyvkfft --verbose --no-cache and copy the details of the output.

If this does not work, please give an exact list of the command used from the line creating the python virtual env to the commands used to test.

kkotyk commented 3 years ago

I figured out my issue. Your suggestions led me to determine that I had some env variables that caused an issue with the installation of pycuda. Once I sorted those out, I got the install to work. One thing I noticed is that installing pyvkfft will auto install the opencl stuff but it doesnt install cython or pycuda as dependencies. Thanks for the help mate!

vincefn commented 3 years ago

Glad to know that helped !

One thing I noticed is that installing pyvkfft will auto install the opencl stuff but it doesnt install cython or pycuda as dependencies

Hm, strange (I assume you mean pyopencl and pycuda - cython is not used) - when I install it does list both packages in the requirements (it would only skip pycuda if installation of the cuda extension failed):

(py36-env) (base) hack:~ vincent$ pip install git+https://github.com/vincefn/pyvkfft.git
Collecting git+https://github.com/vincefn/pyvkfft.git
  Cloning https://github.com/vincefn/pyvkfft.git to /private/var/folders/qw/bqj0rhjx27g5zf6v_bywn1jw0000gn/T/pip-req-build-j5i9zmbj
  Running command git clone -q https://github.com/vincefn/pyvkfft.git /private/var/folders/qw/bqj0rhjx27g5zf6v_bywn1jw0000gn/T/pip-req-build-j5i9zmbj
Requirement already satisfied: numpy in ./dev/py36-env/lib/python3.6/site-packages (from pyvkfft==2021.1b6) (1.19.5)
Requirement already satisfied: pycuda in ./dev/py36-env/lib/python3.6/site-packages (from pyvkfft==2021.1b6) (2020.1)
Requirement already satisfied: pyopencl in ./dev/py36-env/lib/python3.6/site-packages (from pyvkfft==2021.1b6) (2021.1.4)
Requirement already satisfied: mako in ./dev/py36-env/lib/python3.6/site-packages (from pycuda->pyvkfft==2021.1b6) (1.0.7)
Requirement already satisfied: appdirs>=1.4.0 in ./dev/py36-env/lib/python3.6/site-packages (from pycuda->pyvkfft==2021.1b6) (1.4.3)
Requirement already satisfied: pytools>=2011.2 in ./dev/py36-env/lib/python3.6/site-packages (from pycuda->pyvkfft==2021.1b6) (2018.5.2)
Requirement already satisfied: decorator>=3.2.0 in ./dev/py36-env/lib/python3.6/site-packages (from pycuda->pyvkfft==2021.1b6) (4.3.0)
Requirement already satisfied: six>=1.8.0 in ./dev/py36-env/lib/python3.6/site-packages (from pytools>=2011.2->pycuda->pyvkfft==2021.1b6) (1.11.0)
Requirement already satisfied: MarkupSafe>=0.9.2 in ./dev/py36-env/lib/python3.6/site-packages (from mako->pycuda->pyvkfft==2021.1b6) (1.0)