install apex

[masudaryuto]

Hi ! I used conda's python 3.7.10 to instrall apex. However, an error occurred when I executed the following command.

Thank you!!

git clone https://github.com/NVIDIA/apex
cd apex && pip install -v --disable-pip-version-check --no-cache-dir --global-option="--cpp_ext" --global-option="--cuda_ext" --global-option="--distributed_adam" --global-option="--deprecated_fused_adam" ./

(one_peace) > cd apex && pip install -v --disable-pip-version-check --no-cache-dir --global-option="--cpp_ext" --global-option="--cuda_ext" --global-option="--distributed_adam" --global-option="--deprecated_fused_adam" ./

  Collecting setuptools
    Using cached setuptools-68.0.0.tar.gz (2.2 MB)
    Getting requirements to build wheel: started
    Getting requirements to build wheel: finished with status 'done'
    Installing backend dependencies: started
    Installing backend dependencies: finished with status 'done'
    Preparing metadata (pyproject.toml): started
    Preparing metadata (pyproject.toml): finished with status 'done'
  Collecting wheel
    Using cached wheel-0.41.2.tar.gz (98 kB)
    Installing build dependencies: started
    Installing build dependencies: finished with status 'done'
    Getting requirements to build wheel: started
    Getting requirements to build wheel: finished with status 'done'
    Preparing metadata (pyproject.toml): started
    Preparing metadata (pyproject.toml): finished with status 'done'
  Building wheels for collected packages: wheel, setuptools
    Building wheel for wheel (pyproject.toml): started
    Building wheel for wheel (pyproject.toml): finished with status 'done'
    Created wheel for wheel: filename=wheel-0.41.2-py3-none-any.whl size=64848 sha256=425e21995412f8b27c79b2c46c082cfc760a5d012aa2f61c66c54d2eeca12c48
    Stored in directory: /root/.cache/pip/wheels/9e/91/e3/cc8043f0331c745691312e5f9f8fc089777e8db5ee2d4dfc78
    Building wheel for setuptools (pyproject.toml): started
    Building wheel for setuptools (pyproject.toml): finished with status 'done'
    Created wheel for setuptools: filename=setuptools-68.0.0-py3-none-any.whl size=804030 sha256=2caad6eb96aaa9e52ab86e5d157010ba5d33a3c7c8830e5f82a08f5701377d58
    Stored in directory: /root/.cache/pip/wheels/a3/c6/e0/3068d92761531b10ed41a2d0ef2d3110b24856a0383c168a5a
  Successfully built wheel setuptools
  Installing collected packages: wheel, setuptools
  Successfully installed setuptools-68.0.0 wheel-0.41.2
  WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
  Installing build dependencies ... done
  Running command Getting requirements to build wheel
  Traceback (most recent call last):
    File "/root/.asdf/installs/python/anaconda3-2021.05/envs/one_peace/lib/python3.7/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 351, in <module>
      main()
    File "/root/.asdf/installs/python/anaconda3-2021.05/envs/one_peace/lib/python3.7/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 333, in main
      json_out['return_val'] = hook(**hook_input['kwargs'])
    File "/root/.asdf/installs/python/anaconda3-2021.05/envs/one_peace/lib/python3.7/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 118, in get_requires_for_build_wheel
      return hook(config_settings)
    File "/tmp/40831480.1.gpua/pip-build-env-up93c9it/overlay/lib/python3.7/site-packages/setuptools/build_meta.py", line 341, in get_requires_for_build_wheel
      return self._get_build_requires(config_settings, requirements=['wheel'])
    File "/tmp/40831480.1.gpua/pip-build-env-up93c9it/overlay/lib/python3.7/site-packages/setuptools/build_meta.py", line 323, in _get_build_requires
      self.run_setup()
    File "/tmp/40831480.1.gpua/pip-build-env-up93c9it/overlay/lib/python3.7/site-packages/setuptools/build_meta.py", line 338, in run_setup
      exec(code, locals())
    File "<string>", line 5, in <module>
  ModuleNotFoundError: No module named 'packaging'
  error: subprocess-exited-with-error

  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> See above for output.

  note: This error originates from a subprocess, and is likely not a problem with pip.
  full command: /root/.asdf/installs/python/anaconda3-2021.05/envs/one_peace/bin/python /root/.asdf/installs/python/anaconda3-2021.05/envs/one_peace/lib/python3.7/site-packages/pip/_vendor/pep517/in_process/_in_process.py get_requires_for_build_wheel /tmp/40831480.1.gpua/tmpdi9wydf7
  cwd: /root/ONE-PEACE/apex
  Getting requirements to build wheel ... error
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

[logicwong]

please install packaging first

pip install packaging

[masudaryuto]

Thank you!! But, I ran it, with the same result.

[logicwong]

Could you share the log? There should be other errors.

[masudaryuto]

Sorry, I tried, but an error occurred in my case.

> pip install packaging
Requirement already satisfied: packaging in /root/.asdf/installs/python/anaconda3-2021.05/envs/one_peace3_7/lib/python3.7/site-packages (23.2)
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv

Next,

> pip install -v --disable-pip-version-check --no-cache-dir --global-option="--cpp_ext" --global-option="--cuda_ext" --global-option="--distributed_adam" --global-option="--deprecated_fused_adam" ./
Using pip 22.3.1 from /root/.asdf/installs/python/anaconda3-2021.05/envs/one_peace3_7/lib/python3.7/site-packages/pip (python 3.7)
WARNING: Implying --no-binary=:all: due to the presence of --build-option / --global-option / --install-option. Consider using --config-settings for more flexibility.
DEPRECATION: --no-binary currently disables reading from the cache of locally built wheels. In the future --no-binary will not influence the wheel cache. pip 23.1 will enforce this behaviour change. A possible replacement is to use the --no-cache-dir option. You can use the flag --use-feature=no-binary-enable-wheel-cache to test the upcoming behaviour. Discussion can be found at https://github.com/pypa/pip/issues/11453
Processing /root/ONE-PEACE/apex
  Running command pip subprocess to install build dependencies
  DEPRECATION: --no-binary currently disables reading from the cache of locally built wheels. In the future --no-binary will not influence the wheel cache. pip 23.1 will enforce this behaviour change. A possible replacement is to use the --no-cache-dir option. You can use the flag --use-feature=no-binary-enable-wheel-cache to test the upcoming behaviour. Discussion can be found at https://github.com/pypa/pip/issues/11453
  Collecting setuptools
    Using cached setuptools-68.0.0.tar.gz (2.2 MB)
    Getting requirements to build wheel: started
    Getting requirements to build wheel: finished with status 'done'
    Installing backend dependencies: started
    Installing backend dependencies: finished with status 'done'
    Preparing metadata (pyproject.toml): started
    Preparing metadata (pyproject.toml): finished with status 'done'
  Collecting wheel
    Using cached wheel-0.41.2.tar.gz (98 kB)
    Installing build dependencies: started
    Installing build dependencies: finished with status 'done'
    Getting requirements to build wheel: started
    Getting requirements to build wheel: finished with status 'done'
    Preparing metadata (pyproject.toml): started
    Preparing metadata (pyproject.toml): finished with status 'done'
 Building wheels for collected packages: wheel, setuptools
    Building wheel for wheel (pyproject.toml): started
    Building wheel for wheel (pyproject.toml): finished with status 'done'
    Created wheel for wheel: filename=wheel-0.41.2-py3-none-any.whl size=64848 sha256=425e21995412f8b27c79b2c46c082cfc760a5d012aa2f61c66c54d2eeca12c48
    Stored in directory: /root/.cache/pip/wheels/9e/91/e3/cc8043f0331c745691312e5f9f8fc089777e8db5ee2d4dfc78
    Building wheel for setuptools (pyproject.toml): started
    Building wheel for setuptools (pyproject.toml): finished with status 'done'
    Created wheel for setuptools: filename=setuptools-68.0.0-py3-none-any.whl size=804030 sha256=c75e8084d5f64e83fdaf6d3ebbc50685b805faf3869a42bce4415cbfd3af75b1
    Stored in directory: /root/.cache/pip/wheels/a3/c6/e0/3068d92761531b10ed41a2d0ef2d3110b24856a0383c168a5a
  Successfully built wheel setuptools
  Installing collected packages: wheel, setuptools
  Successfully installed setuptools-68.0.0 wheel-0.41.2
  WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
  Installing build dependencies ... done
  Running command Getting requirements to build wheel
  Traceback (most recent call last):
    File "/root/.asdf/installs/python/anaconda3-2021.05/envs/one_peace3_7/lib/python3.7/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 351, in <module>
      main()
    File "/root/.asdf/installs/python/anaconda3-2021.05/envs/one_peace3_7/lib/python3.7/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 333, in main
      json_out['return_val'] = hook(**hook_input['kwargs'])
    File "/root/.asdf/installs/python/anaconda3-2021.05/envs/one_peace3_7/lib/python3.7/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 118, in get_requires_for_build_wheel
      return hook(config_settings)
    File "/tmp/40832125.1.gpua/pip-build-env-m08ejfjt/overlay/lib/python3.7/site-packages/setuptools/build_meta.py", line 341, in get_requires_for_build_wheel
      return self._get_build_requires(config_settings, requirements=['wheel'])
    File "/tmp/40832125.1.gpua/pip-build-env-m08ejfjt/overlay/lib/python3.7/site-packages/setuptools/build_meta.py", line 323, in _get_build_requires
      self.run_setup()
    File "/tmp/40832125.1.gpua/pip-build-env-m08ejfjt/overlay/lib/python3.7/site-packages/setuptools/build_meta.py", line 338, in run_setup
      exec(code, locals())
    File "<string>", line 5, in <module>
  ModuleNotFoundError: No module named 'packaging'
  error: subprocess-exited-with-error

  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> See above for output.

  note: This error originates from a subprocess, and is likely not a problem with pip.
  full command: /root/.asdf/installs/python/anaconda3-2021.05/envs/one_peace3_7/bin/python /root/.asdf/installs/python/anaconda3-2021.05/envs/one_peace3_7/lib/python3.7/site-packages/pip/_vendor/pep517/in_process/_in_process.py get_requires_for_build_wheel /tmp/40832125.1.gpua/tmpgibwsqke
  cwd: /root/ONE-PEACE/apex
  Getting requirements to build wheel ... error
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

[masudaryuto]

Hi ! Adding the "site-packages" path to "_in_process.py" solved it for me.

Thank you for your advice !

> vim /root/.asdf/installs/python/anaconda3-2021.05/envs/one_peace3_7/lib/python3.7/site-packages/pip/_vendor/pep517/in_process/_in_process.py"

Next, add path

sys.path.append('/root/.asdf/installs/python/anaconda3-2021.05/envs/one_peace3_7/lib/python3.7/site-packages')