subsai fails to install from pip

[foolishgrunt]

I had subsai up and running on Arch Linux about a week ago, but wiped the venv as part troubleshooting. Now I am unable to reinstall from pip. Below is the command I run from console, and the output I get:

>>pipx install git+https://github.com/abdeladim-s/subsai

Fatal error from pip prevented installation. Full pip output in file:
    /home/andy/.local/state/pipx/log/cmd_2024-05-04_08.06.24_pip_errors.log

pip seemed to fail to build package:
    tqdm

Some possibly relevant errors from pip install:
    ERROR: Ignored the following versions that require a different python version: 0.55.2 Requires-Python <3.5
    ERROR: Could not find a version that satisfies the requirement triton==2.0.0 (from openai-whisper) (from versions: 2.2.0, 2.3.0)
    ERROR: No matching distribution found for triton==2.0.0

Error installing subsai from spec 'git+https://github.com/abdeladim-s/subsai'.

There couldn't have been new dependencies added in the past week, could there?

Here is the pip log file: cmd_2024-05-04_08.06.24.log And the pip errors log file: cmd_2024-05-04_08.06.24_pip_errors.log

I see warnings regarding Python, TQDM, and Triton.

1. If I recall correctly from the last time I installed, the Python version was not an issue.
2. I have installed python-tqdm from AUR, to no effect.
3. I've tried to compile python-triton - so far my computer crashes because of excessive CPU usage, but I'm not convinced this is a necessary library anyway (I have no discrete GPU to leverage, anyway).

What am I missing?

[abdeladim-s]

@foolishgrunt,

You're not missing anything; it's just you're using Arch! 😅 Such issues are to be expected; a lot can go wrong!

Regarding the project, no changes have been made for more than a month. So, just check your system.

From the logs, openai-whisper depends on python-triton, which I believe is the source of the problem. If triton AUR is broken and you don't need any GPU processing, here's what I suggest: try pulling openai-whisper and remove triton from the requirements and install it separately. If that works, pull the project again and remove openai-whisper from the requirements once more and see if that resolves the issue.

[foolishgrunt]

Why don't you tell us how you really feel about Arch... :)

Since this was bothering me so much, I installed Arch clean and tried again, and am experiencing the same issue. So perhaps the issue is new from whisper, rather than from subsai.

At any rate, I'll try a few more things to get python-triton to build. If I can't get it working, I'll try what you suggested.

[foolishgrunt]

I finally got python-triton (2.1.0, as per the version in AUR) to compile and install, and I'm getting all the same errors.

The error log says:

ERROR: Could not find a version that satisfies the requirement triton==2.0.0 (from openai-whisper) (from versions: 2.2.0, 2.3.0)
ERROR: No matching distribution found for triton==2.0.0

...indicating that I may need to downgrade from v2.1.0 to v2.0.0, but openai/whisper's requirements.txt requires triton>=2.0.0, so I don't know where this requirement is coming from. Any ideas?

[abdeladim-s]

Why don't you tell us how you really feel about Arch... :)

I tried the a lot of Linux distros and I like Arch, the problem is that the bleeding edge is not stable and if you are doing any AI/ML stuff, the only option is to use a Debian based distro because it's the only one that is well supported by the tech stack. But anyways, this is not that important, as long as you are using Linux you are good :)

ERROR: Could not find a version that satisfies the requirement triton==2.0.0 (from openai-whisper) (from versions: 2.2.0, 2.3.0) ERROR: No matching distribution found for triton==2.0.0

...indicating that I may need to downgrade from v2.1.0 to v2.0.0, but openai/whisper's requirements.txt requires triton>=2.0.0, so I don't know where this requirement is coming from. Any ideas?

As you can see from the logs, the problem still openai/whisper, which is weird, because you already installed triton 2.1

Have you tried to install openai whisper separately ?

[foolishgrunt]

Yes, pipx install git+https://github.com/openai/whisper.git works, and whisper installs fine.

Afterward, I downloaded the subsai repo, modified requirements.txt to remove the first line (openai-whisper==20231106), then ran pipx install ~/subsai-main. The install still fails, and the last two errors are the same as before - see here: cmd_2024-05-05_18.51.06_pip_errors.log

[abdeladim-s]

If you build triton yourself then you should not use pipx I believe, because I think whenever you run it, it creates a separate virtual environment for the package.

Please get more control over the environment and do the setup manually.

• First install or downgrade to Python 3.11 (Since you are using Arch, probably you are using 3.12).
• create a virtual env with venv and activate it.
• Install the package with pip install git+.

[foolishgrunt]

Phew... finally got it working again! Here were my steps

1. Created my venv (using Arch's default Python package, 3.12.3)
2. Ran pip install git+https://github.com/abdeladim-s/subsai, got a very familiar set of errors
3. Ran pip install triton, installation succeeded
4. Ran pip install git+https://github.com/abdeladim-s/subsai, errors remained unchanged
5. Wiped venv and created afresh using Python 3.11.9
6. Ran pip install git+https://github.com/abdeladim-s/subsai, installation succeeded

It seems like the issue all along may have been the Python version. And looking back, the Arch repos bumped the version from 3.12.2 to 3.12.3 on 27 April, which was right around when my issues began. So it's not a subsai bug, or even a whisper bug.

Once again, thank you for your assistance in identifying the issue - I very much appreciate your help and your responsiveness!