Open Renzzauw opened 1 year ago
Update: This issue does not occur when I check out an older commit (e.g. commit associated with #477), perhaps this bug got introduced in a recent commit.
Hmm, so the culprit seems to be #478. Can you try to run pip install -U pip wheel setuptools
and then repeat the installation? The only thing we changed in https://github.com/NVIDIA/VideoProcessingFramework/commit/ebbf718d8bfc19bd84c9ede342060ac6c1c3c2f5 is to add this snippet to PyTorchNVCodec https://github.com/NVIDIA/VideoProcessingFramework/blob/master/src/PytorchNvCodec/pyproject.toml.
@theHamsta thanks for your reply, I just updated those pip packages and repeated the installation, but I still get the same exceptions.
Can you see anything suspicious in the diff I linked? Would for instance deleting https://github.com/NVIDIA/VideoProcessingFramework/blob/master/src/PytorchNvCodec/pyproject.toml make a difference for you. This would be one of the few differences between the commits you said that would work and that would not work
Hi @theHamsta, I have my suspicions that my issues are not related to the changes made in the latest commit. When I started with a fresh virtual environment, I did manage to successfully install the latest commit and import the packages with no issues. I'll investigate further if I run into any issues, but I'll close the issue for now. Thanks for the help!
https://github.com/NVIDIA/VideoProcessingFramework/blob/master/src/PytorchNvCodec/pyproject.toml seems indeed cause a problem. Since the isolate installation of torch
might lead to a version mismatch of the compile-torch and the runtime-torch. If both are updated to the newest version there is no problem.
I think I found the reason. When I try to reinstall PytorchNvCodec and enter pip uninstall PytorchNvCodec
and I saw this shown
Found existing installation: PytorchNvCodec 0.0.0 Uninstalling PytorchNvCodec-0.0.0: Would remove: /home/aque/.conda/envs/3.9/lib/python3.9/site-packages/PytorchNvCodec-0.0.0.dist-info/* /home/aque/.conda/envs/3.9/lib/python3.9/site-packages/PytorchNvCodec/* /home/aque/.conda/envs/3.9/lib/python3.9/site-packages/_PytorchNvCodec.cpython-39-x86_64-linux-gnu.so
so actually it copied the .so file into a wrong path, and after I copy it to the PytorchNvCodec path the problem solved
Recreated this issue with the NVIDIA Pytorch container image.
You can reproduce with the following Dockerfile:
FROM nvcr.io/nvidia/pytorch:23.05-py3
RUN git clone https://git.videolan.org/git/ffmpeg/nv-codec-headers.git
RUN cd nv-codec-headers && make install && cd -
RUN git clone https://git.ffmpeg.org/ffmpeg.git ffmpeg/ && cd ffmpeg
RUN apt-get update && apt-get install -y build-essential yasm cmake libtool libc6 libc6-dev unzip wget libnuma1 libnuma-dev
RUN cd ffmpeg && ./configure --enable-nonfree --enable-cuda-nvcc --enable-libnpp --extra-cflags=-I/usr/local/cuda/include --extra-ldflags=-L/usr/local/cuda/lib64 --disable-static --enable-shared
RUN cd ffmpeg && make -j8
RUN cd ffmpeg && make install
RUN git clone https://github.com/NVIDIA/VideoProcessingFramework
RUN apt install -y libavfilter-dev libavformat-dev libavcodec-dev libswresample-dev libavutil-dev wget build-essential
RUN pip install --upgrade pip
RUN cd VideoProcessingFramework && pip install .
RUN cd VideoProcessingFramework && pip install src/PytorchNvCodec
RUN mkdir /app
Running python in the container and then trying to import PytorchNvCodec generates an error:
>>> import PytorchNvCodec
Traceback (most recent call last):
File "/usr/local/lib/python3.10/dist-packages/PytorchNvCodec/__init__.py", line 18, in <module>
from _PytorchNvCodec import *
ImportError: /usr/local/lib/python3.10/dist-packages/_PytorchNvCodec.cpython-310-x86_64-linux-gnu.so: undefined symbol: _ZN3c106detail14torchCheckFailEPKcS2_jRKSs
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python3.10/dist-packages/PytorchNvCodec/__init__.py", line 22, in <module>
raise RuntimeError("Failed to import native module _PytorchNvCodec! "
RuntimeError: Failed to import native module _PytorchNvCodec! Please check whether "/usr/local/lib/python3.10/dist-packages/PytorchNvCodec/_PytorchNvCodec.cpython-310-x86_64-linux-gnu.so" exists and can find all library dependencies (CUDA, ffmpeg).
On Unix systems, you can use `ldd` on the file to see whether it can find all dependencies.
On Windows, you can use "dumpbin /dependents" in a Visual Studio command prompt or
https://github.com/lucasg/Dependencies/releases.
Again, deleting the pyproject.toml
fixes the build, and makes the import work. Change the line to delete pyproject.toml
before the PytorchNvCodec buidl:
RUN cd VideoProcessingFramework && rm src/PytorchNvCodec/pyproject.toml && pip install src/PytorchNvCodec
@onmygame same problem .. how to solve ...
I think I found the reason. When I try to reinstall PytorchNvCodec and enter
pip uninstall PytorchNvCodec
and I saw this shownFound existing installation: PytorchNvCodec 0.0.0 Uninstalling PytorchNvCodec-0.0.0: Would remove: /home/aque/.conda/envs/3.9/lib/python3.9/site-packages/PytorchNvCodec-0.0.0.dist-info/* /home/aque/.conda/envs/3.9/lib/python3.9/site-packages/PytorchNvCodec/* /home/aque/.conda/envs/3.9/lib/python3.9/site-packages/_PytorchNvCodec.cpython-39-x86_64-linux-gnu.so
so actually it copied the .so file into a wrong path, and after I copy it to the PytorchNvCodec path the problem solved
Hi, I happened to run into this same issue. I then copied _PytorchNvCodec.cpython-39-x86_64-linux-gnu.so
under my site-packages/PytorchNvCodec
folder, specifically here home/user/anaconda3/envs/test_env/lib/python3.10/site-packages/PytorchNvCodec
but the issue still persists!
Here is my system info:
OS: Ubuntu 22.04 on WSL2 (Windows 11)
CUDA: 11.8
Cudnn: 9.0
Torch: 2.0.1+cu117
And when I run ldd _PytorchNvCodec.cpython-310-x86_64-linux-gnu.so
this is what I see:
linux-vdso.so.1 (0x00007ffcd7ffe000)
libc10.so => not found
libtorch_cpu.so => not found
libtorch_python.so => not found
libcudart.so.11.0 => /usr/local/cuda/lib64/libcudart.so.11.0 (0x00007fc806400000)
libstdc++.so.6 => /home/z84294149/anaconda3/envs/gauss_view/lib/libstdc++.so.6 (0x00007fc8061ec000)
libgcc_s.so.1 => /home/z84294149/anaconda3/envs/gauss_view/lib/libgcc_s.so.1 (0x00007fc806800000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fc805fc3000)
/lib64/ld-linux-x86-64.so.2 (0x00007fc806849000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fc8067fb000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fc8067f6000)
librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007fc8067f1000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fc806708000)
I ran into the same issue, too. Finally it was resolved by python3.10 and the newest version of torch downloaded in it.
Hi!
I'm trying to use the PyTorch plugin in my project. I've installed the latest version as follows (as partly described by the comments in the SamplePytorch.py example):
Upon trying to import as follows:
I get the following exceptions:
When I check the files in my virtual environment site packages, the folder only contains an
__init__.py
file, but no.so
files, so some files seem to be missing here that are being referred to by the exception.Perhaps this is a bug during the pip installation process?
Thanks in advance for the help!