Unable to install ffcv via pip

[kerneeee]

Unfortunately i am not able to install ffcv via pip in my environment. Following exceptions is raised during installation:

ERROR: Command errored out with exit status 1: command: /usr/bin/python3 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-bb88nk1n/ffcv/setup.py'"'"'; __file__='"'"'/tmp/pip-install-bb88nk1n/ffcv/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-install-bb88nk1n/ffcv/pip-egg-info cwd: /tmp/pip-install-bb88nk1n/ffcv/ Complete output (7 lines): Traceback (most recent call last): File "<string>", line 1, in <module> File "/tmp/pip-install-bb88nk1n/ffcv/setup.py", line 30, in <module> extension_kwargs = pkgconfig('opencv4', extension_kwargs) File "/tmp/pip-install-bb88nk1n/ffcv/setup.py", line 18, in pkgconfig raise Exception() Exception

The error may points to a missing installation of opencv4. But actually i have opencv-python==4.6.0.66 installed. Am i missing anything here?

[zero0kiriyu]

I have the same problem while installing the ffcv, just using the conda install opencv -c conda-forge would solve this problem.

[kerneeee]

thanks for the reply. acually i can't use conda, therefore another approach would be interesting.

[richard-vock]

I have the same problem using poetry:

 × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> [8 lines of output]
    Traceback (most recent call last):
      File "<string>", line 2, in <module>
      File "<pip-setuptools-caller>", line 34, in <module>
      File "/tmp/pip-req-build-mz7uq447/setup.py", line 30, in <module>
        extension_kwargs = pkgconfig('opencv4', extension_kwargs)
      File "/tmp/pip-req-build-mz7uq447/setup.py", line 18, in pkgconfig
        raise Exception()
    Exception
    [end of output]

[richard-vock]

In my case a

apt install libturbojpeg0-dev libopencv-dev

(before pip/poetry install) fixed it. Note that installing libopencv-dev alone did not suffice.

[Batwho]

A similar problem using Windows

 Collecting ffcv
  Using cached ffcv-0.0.3.tar.gz (53 kB)
  Preparing metadata (setup.py) ... error
  error: subprocess-exited-with-error

  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> [8 lines of output]
      Traceback (most recent call last):
        File "<string>", line 2, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "C:\AppData\Local\Temp\pip-install-i73a9x2t\ffcv_a8d4f1ce88c64d6eb9631b9e8ef1d164\setup.py", line 30, in <module>
          extension_kwargs = pkgconfig('opencv4', extension_kwargs)
        File "C:\AppData\Local\Temp\pip-install-i73a9x2t\ffcv_a8d4f1ce88c64d6eb9631b9e8ef1d164\setup.py", line 18, in pkgconfig
          raise Exception()
      Exception
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

[woodenchild]

@@zero0kiriyu @richard-vock I execute conda install opencv libopencv libjpeg-turbo while it still fails to install ffcv with the same errors. I have no root to do apt install or sudo apt install. Is there any other way？

[woodenchild]

@richard-vock @zero0kiriyu @Batwho @kerneeee @lengstrom It works when I use the setting in https://pypi.org/project/ffcv/.

[Batwho]

@woodenchild Hi, Thank you for the suggestion. I wonder if you are using the setting as conda create -y -n ffcv python=3.9 cupy pkg-config compilers libjpeg-turbo opencv pytorch torchvision cudatoolkit=11.3 numba -c pytorch -c conda-forge conda activate ffcv pip install ffcv because I think this is the same setting as the GitHub code provided.

[XksA-me]

In my case a

apt install libturbojpeg0-dev libopencv-dev

(before pip/poetry install) fixed it. Note that installing libopencv-dev alone did not suffice.

Thank you. This is effective. Complete steps: The virtual environment I created with conda is called myconda, python 3.9, cuda11.3, and I installed torch 1.12.1,

conda create -n myconda python=3.9
conda activate myconda
pip install torch==1.12.1+cu113 torchvision==0.13.1+cu113 torchaudio==0.12.1 --extra-index-url  https://download.pytorch.org/whl/cu113

You can rent a machine in matpool.com and directly select the Python 1.12.1 environment for the above basic environment. They should also add an ffcv system environment based on this environment later.

The first step to install the system depends on:

apt update
apt install pkg-config libturbojpeg0-dev libopencv-dev -y

Step 2: Install Python dependencies:

pip install cupy-cuda113 numba

Step 3: Add environment variables (to~/. bashrc):

export LD_ LIBRARY_ PATH=/root/miniconda3/envs/myconda/lib:$LD_ LIBRARY_ PATH

Step 4:

source  ~/.bashrc
pip install ffcv

[kschuerholt]

Thanks for the pointers! I had the same problem. In my case, building on top of the nvidia/pytorch docker, installing libopencv-dev breaks cuda support, so it's unfortunately not an option. Conda also doesn't really work that well in our compute environment.

I ended up finding a way by building opencv from source with cuda support enabled. The Dockerfile can be found here. It took me a while to figure this out, so I hope it helps someone getting there faster.

[andrewilyas]

Thanks! I added a link to this in the README.md