bigscience-workshop / Megatron-DeepSpeed

Ongoing research training transformer language models at scale, including: BERT & GPT-2
Other
1.3k stars 211 forks source link

ModuleNotFoundError: No module named 'packaging' when install apex #390

Closed SeekPoint closed 1 year ago

SeekPoint commented 1 year ago

(gh_Megatron-DeepSpeed) amd00@MZ32-00:~/llm_dev/Megatron-DeepSpeed/apex$ python -m pip install --global-option="--cpp_ext" --global-option="--cuda_ext" --no-cache -v --disable-pip-version-check . 2>&1 | tee build.log Using pip 23.1.2 from /home/amd00/anaconda3/envs/gh_Megatron-DeepSpeed/lib/python3.10/site-packages/pip (python 3.10) DEPRECATION: --build-option and --global-option are deprecated. pip 23.3 will enforce this behaviour change. A possible replacement is to use --config-settings. Discussion can be found at https://github.com/pypa/pip/issues/11859 WARNING: Implying --no-binary=:all: due to the presence of --build-option / --global-option. Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple Processing /home/amd00/llm_dev/Megatron-DeepSpeed/apex Installing build dependencies: started Running command pip subprocess to install build dependencies Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple Collecting setuptools Downloading https://pypi.tuna.tsinghua.edu.cn/packages/dc/98/5f896af066c128669229ff1aa81553ac14cfb3e5e74b6b44594132b8540e/setuptools-68.0.0.tar.gz (2.2 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.2/2.2 MB 5.8 MB/s eta 0:00:00 Getting requirements to build wheel: started Getting requirements to build wheel: finished with status 'done' Installing backend dependencies: started Installing backend dependencies: finished with status 'done' Preparing metadata (pyproject.toml): started Preparing metadata (pyproject.toml): finished with status 'done' Collecting wheel Using cached wheel-0.40.0-py3-none-any.whl Building wheels for collected packages: setuptools Building wheel for setuptools (pyproject.toml): started Building wheel for setuptools (pyproject.toml): finished with status 'done' Created wheel for setuptools: filename=setuptools-68.0.0-py3-none-any.whl size=804037 sha256=30608e057b2f6534956be9e9bf6df511597714f85c79a5ca590150956a6c249f Stored in directory: /home/amd00/.cache/pip/wheels/4a/c0/d0/d1d7bdfed20eba7e9acbaa4135e8b87c2ae7ea6d6bfa1dc7d1 Successfully built setuptools Installing collected packages: wheel, setuptools ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. spyder 5.4.3 requires pyqt5<5.16, which is not installed. spyder 5.4.3 requires pyqtwebengine<5.16, which is not installed. Successfully installed setuptools-68.0.0 wheel-0.40.0 Installing build dependencies: finished with status 'done' Getting requirements to build wheel: started Running command Getting requirements to build wheel Traceback (most recent call last): File "/home/amd00/anaconda3/envs/gh_Megatron-DeepSpeed/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in main() File "/home/amd00/anaconda3/envs/gh_Megatron-DeepSpeed/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main json_out['return_val'] = hook(**hook_input['kwargs']) File "/home/amd00/anaconda3/envs/gh_Megatron-DeepSpeed/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 118, in get_requires_for_build_wheel return hook(config_settings) File "/tmp/pip-build-env-5hee3xv2/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 341, in get_requires_for_build_wheel return self._get_build_requires(config_settings, requirements=['wheel']) File "/tmp/pip-build-env-5hee3xv2/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 323, in _get_build_requires self.run_setup() File "/tmp/pip-build-env-5hee3xv2/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 338, in run_setup exec(code, locals()) File "", line 4, in ModuleNotFoundError: No module named 'packaging' error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully. │ exit code: 1 ╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip. full command: /home/amd00/anaconda3/envs/gh_Megatron-DeepSpeed/bin/python /home/amd00/anaconda3/envs/gh_Megatron-DeepSpeed/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py get_requires_for_build_wheel /tmp/tmp0nflxl7a cwd: /home/amd00/llm_dev/Megatron-DeepSpeed/apex Getting requirements to build wheel: finished with status 'error' error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully. │ exit code: 1 ╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip. (gh_Megatron-DeepSpeed) amd00@MZ32-00:~/llm_dev/Megatron-DeepSpeed/apex$

SeekPoint commented 1 year ago

I done the pip install packaging or apt install

SeekPoint commented 1 year ago

I do it under the conda evn and failed

but I can make it with root

JumpingRain commented 1 year ago

I do it under the conda evn and failed

but I can make it with root

how do you slove it