Can't install. Error

[AfterHAL]

Hi. I've been trying to install torch-cubas-hgemm (for flux-fp8-api) on ubuntu without success.

The same error ModuleNotFoundError: No module named 'torch', whether with pip install git+https://github.com/aredden/torch-cublas-hgemm.git@master or with python -m pip install -U -v . (after cloning the repository).

--> Of course, torch is installed.

Regards.

python -m pip install -U -v .

Using pip 24.2 from /mnt/w/TDocker/gradio1/flux-fp8-api/venv/lib/python3.12/site-packages/pip (python 3.12)
Processing /mnt/w/TDocker/gradio1/flux-fp8-api/torch-cublas-hgemm
  Running command pip subprocess to install build dependencies
  Using pip 24.2 from /mnt/w/TDocker/gradio1/flux-fp8-api/venv/lib/python3.12/site-packages/pip (python 3.12)
  Collecting setuptools>=40.8.0
    Obtaining dependency information for setuptools>=40.8.0 from https://files.pythonhosted.org/packages/ff/ae/f19306b5a221f6a436d8f2238d5b80925004093fa3edea59835b514d9057/setuptools-75.1.0-py3-none-any.whl.metadata
    Using cached setuptools-75.1.0-py3-none-any.whl.metadata (6.9 kB)
  Using cached setuptools-75.1.0-py3-none-any.whl (1.2 MB)
  Installing collected packages: setuptools
  Successfully installed setuptools-75.1.0
  Installing build dependencies ... done
  Running command Getting requirements to build wheel
  Traceback (most recent call last):
    File "/mnt/w/TDocker/gradio1/flux-fp8-api/venv/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
      main()
    File "/mnt/w/TDocker/gradio1/flux-fp8-api/venv/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
      json_out['return_val'] = hook(**hook_input['kwargs'])
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/mnt/w/TDocker/gradio1/flux-fp8-api/venv/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 118, in get_requires_for_build_wheel
      return hook(config_settings)
             ^^^^^^^^^^^^^^^^^^^^^
    File "/tmp/pip-build-env-a9kwdlcq/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 332, in get_requires_for_build_wheel
      return self._get_build_requires(config_settings, requirements=[])
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/tmp/pip-build-env-a9kwdlcq/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 302, in _get_build_requires
      self.run_setup()
    File "/tmp/pip-build-env-a9kwdlcq/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 503, in run_setup
      super().run_setup(setup_script=setup_script)
    File "/tmp/pip-build-env-a9kwdlcq/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 318, in run_setup
      exec(code, locals())
    File "<string>", line 3, in <module>
  ModuleNotFoundError: No module named 'torch'
  error: subprocess-exited-with-error

  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> See above for output.

  note: This error originates from a subprocess, and is likely not a problem with pip.
  full command: /mnt/w/TDocker/gradio1/flux-fp8-api/venv/bin/python /mnt/w/TDocker/gradio1/flux-fp8-api/venv/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py get_requires_for_build_wheel /tmp/tmpnp9reoxk
  cwd: /mnt/w/TDocker/gradio1/flux-fp8-api/torch-cublas-hgemm
  Getting requirements to build wheel ... error
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

[aredden]

That is very strange- the only thing I can think of would be that python isn't pointing to the right python. You're sure that when you do python -m pip list | grep torch - torch shows up?

[AfterHAL]

That is very strange- the only thing I can think of would be that python isn't pointing to the right python. You're sure that when you do python -m pip list | grep torch - torch shows up?

Hi @aredden. Thanks for helping out. Nothing special for me about the torch installation :

python -m pip list | grep torch
pytorch-triton           3.1.0+5fe38ffd73
torch                    2.6.0.dev20240917+cu124
torchaudio               2.5.0.dev20240917+cu124
torchvision              0.20.0.dev20240917+cu124

I'm trying to run flux-fp8-api from a fresh virtual environment. I've been trying updating and checking everything I could thing of without success.

By the way, I'm looking for way to use a fast loading Flux-dev-fp8 model like the "easy to use ComfyUI fp8 checkpoint for the Flux dev (safetensors file here)" of ComfyUI (with the --fast argument). It loads in 30 to 40 seconds on my 64GBram/4090-24GBvram PC, and generates a 1280x1280px image in less then 20 secs, even with 4 or 5 LoRAs. This setup uses around 20 GBvram. I've been looking inside the the code of ComfyUI and the official Flux.1 repository, but it is a little too hard for me.

You're code looks good and I would like to know how to do it, but I'm stuck ;)

Let me know if you have any idea. Thanks again.

[AfterHAL]

Hi @aredden. I finally started from scratch with a fresh installation of Ubuntu and it's ok now. By the way, I don't know the origin of the error I had before.

[aredden]

Ah okay cool! Yeah I'm unsure either. I'm glad you got it sorted :)

[MichaelP84]

I have the exact same issue, so let me know if you get any clarity on it, thanks!

[mikhail-shevtsov-wiregate]

Just in case if someone faces this issue:

python -m pip install --no-build-isolation -U -v .

Adding --no-build-isolation solved the problem.

[spejamas]

I think this happens because the package depends on torch/cuda during its setup.py execution, not only at runtime. --no-build-isolation will work for some (by allowing the installation to look globally for torch/cuda I guess?). Some environments though—like when trying to deploy with some cloud providers—won't have torch/cuda available during the build phase, so it won't work for everyone.

[fyepi]

I have encountered the same problem when I was trying flux-fp8-api.

I used python -m venv venv to create a virtual environment, activated it and then followed the installation steps to install torch then install requirements.txt then this problem happened.

I used ubuntu 22.04 and python 3.12. To reinstall ubuntu is way too costly for me.

@spejamas
Some environments though—like when trying to deploy with some cloud providers—won't have torch/cuda available during the build phase seems to be the root cause. Any way to get torch ready in build phase?

Thanks.