Closed exnx closed 2 years ago
help wanted ! Collecting ffcv Using cached ffcv-0.0.3.tar.gz (53 kB) Preparing metadata (setup.py) ... error error: subprocess-exited-with-error
× python setup.py egg_info did not run successfully.
│ exit code: 1
╰─> [6 lines of output]
Traceback (most recent call last):
File "<string>", line 2, in <module>
File "<pip-setuptools-caller>", line 34, in <module>
File "C:\Users\ASUS\AppData\Local\Temp\pip-install-fzrd9cqe\ffcv_549c561025204b5bb7558015a0e2dbcc\setup.py", line 33, in <module>
extension_kwargs['libraries'].append('pthread')
KeyError: 'libraries'
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed
× Encountered error while generating package metadata.
╰─> See above for output.
note: This is an issue with the package mentioned above, not pip.
hint: See above for details.
i have probably done everything . . . in fact I have uninstalled anaconda and again installed and tried . . . everytime getting same issue.
I have followed :
1) conda create -n ffcv python=3.9 cupy pkg-config compilers libjpeg-turbo opencv pytorch torchvision cudatoolkit=11.3 numba -c pytorch -c conda-forge
2) conda activate ffcv
3) pip install ffcv
Hi @exnx:
What are the output for
pkg-config --cflags --libs opencv4
pkg-config --cflags --libs libturbojpeg
?
@bibhabasumohapatra We do not support Windows. There was a pull request that tried to add support but if I recall properly it wasn't really working
@GuillaumeLeclerc
Ah, it can't find opencv. It did find libturbojpeg ok.
pkg-config --cflags --libs opencv4
output:
Package opencv4 was not found in the pkg-config search path.
Perhaps you should add the directory containing opencv4.pc
to the PKG_CONFIG_PATH environment variable
No package 'opencv4' found
pkg-config --cflags --libs libturbojpeg
output:
-I/home/miniconda3/envs/hippo_fccv_clone/include -L/home/miniconda3/envs/hippo_fccv_clone/lib -lturbojpeg
I used pip install for opencv, because for some reason I can't conda install in this existing environment, something to do with inconsistencies. Is conda the only way to install opencv, or I guess the from source route as well?
Thanks!
The pip
version is a built binary and doesn't contain the header files needed for the compilation of ffcv
. There are more simpler ways than from source but they usually depend on your operating system. For example on ubuntu that might be apt install libopencv-dev
or something along those lines. In some cases the OS installs it under opencv
instead of opencv4
so you might have to either update the setup.py script or update the pc
file in your OS folder.
Keep us updated with your progress
ok, my lab mate solved our issue. Our procedure was:
make a fresh conda env first on it's own entirely, using python 3.8. For whatever reason, doing this separately instead of creating the environment and installing the ffcv dependancies in step 2 made a difference. Things broke in step 4 later if I didn't start with this.
now, conda install the ffcv requirements with this below:
conda install cupy pkg-config compilers libjpeg-turbo opencv pytorch torchvision torchaudio cudatoolkit=11.3 numba -c pytorch -c conda-forge
(This used pytorch 1.10 for me)
pip install ffcv
finally, pip install my other project's requirements (e.g., pytorch_lightning, etc)
This workflow resolves my issue, no need to install opencv from source etc.
Ok sounds good!
Note that pytorch_ligthing currently has issues with FFCV as it tries to preload the data again when FFCV has already done everything perfectly. We unfortunately don't recommend using them together as of now :/ There is a temporary solution somewhere on our Slack if you want to try though
Thanks for the heads up, yes I'd love to try this hack. I'll join it now.
Hi, I have an existing conda environment that I'm trying to add ffcv to. Oddly enough, if I create a new conda environment with ffcv in it (via the conda create line given in the docs), I can import ffcv just fine. However, if I want to add the ffcv dependancies to an existing conda environment, ffcv won't install. Here's my work flow.
(Note: I used pip instead of conda install because I get some
Then I get this exception:
Note: I used pip because when I tried to use conda for (cupy, pkg-config, compilers, open-cv, etc), I get this error:
Which after a long output of conflicts, it states this:
Your installed version is: 2.27
To get around this, I used pip, and didn't have these conflicts. But again, I can't install ffcv. Anybody have any idea what might be the issue?
Update:
I've also tried install opencv from source via https://docs.opencv.org/4.x/d7/d9f/tutorial_linux_install.html#tutorial_linux_install_quick_build_core, but still got the same exception above when trying to pip install ffcv
Update2:
I cloned the ffcv repo and ran python setup.py, and inspected the
extension_kwargs
var, and got this: