HITsz-TMG / UMOE-Scaling-Unified-Multimodal-LLMs

The codes about "Uni-MoE: Scaling Unified Multimodal Models with Mixture of Experts"
https://uni-moe.github.io/
728 stars 33 forks source link

Upgrade Uni_Moe to support LLaVa correctly #7

Open Bhagyashreet20 opened 2 weeks ago

Bhagyashreet20 commented 2 weeks ago

LLaVA requires 3.10 python (https://github.com/haotian-liu/LLaVA) and majority of containers are only with 3.10.x. Python 3.9.x is not a stable version for long term. Given all these, it is fundamental for upgrading Uni_Moe to 3.10.x ecosystem asap. It is non-usable otherwise.

Errors reported if not upgraded when install from Uni_Moe

pip install -r env.txt
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Obtaining llava from git+https://github.com/haotian-liu/LLaVA.git@e61aa3f88f58f8e871b9c2476d743724e271c776#egg=llava (from -r env.txt (line 83))
  Skipping because already up-to-date.
  Installing build dependencies ... error
  error: subprocess-exited-with-error

  × pip subprocess to install build dependencies did not run successfully.
  │ exit code: 1
  ╰─> [7 lines of output]
      Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com, https://pypi.ngc.nvidia.com
      Collecting setuptools>=61.0
        Downloading setuptools-70.1.0-py3-none-any.whl.metadata (6.0 kB)
      Downloading setuptools-70.1.0-py3-none-any.whl (882 kB)
         ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 882.6/882.6 kB 24.1 MB/s eta 0:00:00
      Installing collected packages: setuptools
      ERROR: Cannot set --home and --prefix together
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× pip subprocess to install build dependencies did not run successfully.
│ exit code: 1
╰─> See above for output.
Bhagyashreet20 commented 2 weeks ago

In Uni_Moe_v2 another messed up dependencies:

pip install tutel git+https://github.com/microsoft/tutel@56dbd664341cf6485c9fa292955f77d3ac918a65
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Collecting git+https://github.com/microsoft/tutel@56dbd664341cf6485c9fa292955f77d3ac918a65
  Cloning https://github.com/microsoft/tutel (to revision 56dbd664341cf6485c9fa292955f77d3ac918a65) to /tmp/pip-req-build-ze4vk9_1
  Running command git clone --filter=blob:none --quiet https://github.com/microsoft/tutel /tmp/pip-req-build-ze4vk9_1
  Running command git rev-parse -q --verify 'sha^56dbd664341cf6485c9fa292955f77d3ac918a65'
  Running command git fetch -q https://github.com/microsoft/tutel 56dbd664341cf6485c9fa292955f77d3ac918a65
  Running command git checkout -q 56dbd664341cf6485c9fa292955f77d3ac918a65
  Resolved https://github.com/microsoft/tutel to commit 56dbd664341cf6485c9fa292955f77d3ac918a65
  Preparing metadata (setup.py) ... done
ERROR: Could not find a version that satisfies the requirement tutel (from versions: none)
ERROR: No matching distribution found for tutel
Bhagyashreet20 commented 2 weeks ago

Another README dependency messed up

$ pip install VideoFileClip
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
ERROR: Could not find a version that satisfies the requirement VideoFileClip (from versions: none)
ERROR: No matching distribution found for VideoFileClip
expapa commented 2 weeks ago

So sorry for the mistakes in (https://github.com/HITsz-TMG/UMOE-Scaling-Unified-Multimodal-LLMs/blob/master/Uni_MoE_v2/README.md) and (https://github.com/HITsz-TMG/UMOE-Scaling-Unified-Multimodal-LLMs/blob/master/Uni_MoE_v2/env.txt), LLaVA and tutel is currently not in use by Uni-MoE, so these dependencies will not influence the process of Uni-MoE code whether they are installed or not. As for VideoFileClip, this dependency can be download by "pip install moviepy" which is now updated in README file. Thank you so much for the reminder and your support of our model.

Bhagyashreet20 commented 2 weeks ago

thanks! Also, can you please upgrade all your scripts to more stable python version that is compatible with llava (3.10 python version)?

expapa commented 1 week ago

will do :)