search

giampaolo / psutil

Cross-platform lib for process and system monitoring in Python
BSD 3-Clause "New" or "Revised" License
10.11k stars 1.37k forks source link

[Ubuntu] ERROR: Failed building wheel for psutil #2301

Closed BillRaymond closed 10 months ago

BillRaymond commented 10 months ago

Summary

Description

I am building a Docker container to use JupyterLab, which fails to build due to an error with psutil, so I attempted to install pustil first and that appears to be the issue. I install Python using the deadsnakes PPA, like this:

ARG PYVER="3.11"
RUN apt-get install -y software-properties-common && \
    add-apt-repository -y ppa:deadsnakes/ppa && \
    apt-get install -y python$PYVER \
    python3-pip

Next, I upgrade/update my package managers. I also make sure 3.11.5 (which is the most current version of 3.11.x at the time of this writing), like this:

# Upgrade packages to the latest version
RUN apt-get -y upgrade && \
    python3 -m pip install --upgrade pip

# Set PYVER as the default Python interpreter
RUN update-alternatives --install /usr/bin/python3 python /usr/bin/python$PYVER 1
RUN update-alternatives --set python /usr/bin/python$PYVER

I install gcc, musl-dev and python3-dev, like this: 

apt-get install -y gcc musl-dev python$PYVER-dev

Whether I use pip install psutil or the more detailed version, which is pip install --no-binary :all: psutil, psutil never installs.

Here is the detailed log when using pip install --no-binary :all: psutil:

root@db974e421c12:/workspaces/my-ai-docker-image# pip install --no-binary :all: psutil
Collecting psutil
  Using cached psutil-5.9.5.tar.gz (493 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: psutil
  Building wheel for psutil (pyproject.toml) ... error
  error: subprocess-exited-with-error

  × Building wheel for psutil (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [106 lines of output]
      running bdist_wheel
      running build
      running build_py
      creating build
      creating build/lib.linux-aarch64-3.11
      creating build/lib.linux-aarch64-3.11/psutil
      copying psutil/_psposix.py -> build/lib.linux-aarch64-3.11/psutil
      copying psutil/__init__.py -> build/lib.linux-aarch64-3.11/psutil
      copying psutil/_psbsd.py -> build/lib.linux-aarch64-3.11/psutil
      copying psutil/_psosx.py -> build/lib.linux-aarch64-3.11/psutil
      copying psutil/_pswindows.py -> build/lib.linux-aarch64-3.11/psutil
      copying psutil/_pssunos.py -> build/lib.linux-aarch64-3.11/psutil
      copying psutil/_psaix.py -> build/lib.linux-aarch64-3.11/psutil
      copying psutil/_pslinux.py -> build/lib.linux-aarch64-3.11/psutil
      copying psutil/_common.py -> build/lib.linux-aarch64-3.11/psutil
      copying psutil/_compat.py -> build/lib.linux-aarch64-3.11/psutil
      creating build/lib.linux-aarch64-3.11/psutil/tests
      copying psutil/tests/test_bsd.py -> build/lib.linux-aarch64-3.11/psutil/tests
      copying psutil/tests/test_sunos.py -> build/lib.linux-aarch64-3.11/psutil/tests
      copying psutil/tests/test_system.py -> build/lib.linux-aarch64-3.11/psutil/tests
      copying psutil/tests/test_misc.py -> build/lib.linux-aarch64-3.11/psutil/tests
      copying psutil/tests/test_memleaks.py -> build/lib.linux-aarch64-3.11/psutil/tests
      copying psutil/tests/__init__.py -> build/lib.linux-aarch64-3.11/psutil/tests
      copying psutil/tests/__main__.py -> build/lib.linux-aarch64-3.11/psutil/tests
      copying psutil/tests/test_windows.py -> build/lib.linux-aarch64-3.11/psutil/tests
      copying psutil/tests/test_posix.py -> build/lib.linux-aarch64-3.11/psutil/tests
      copying psutil/tests/test_process.py -> build/lib.linux-aarch64-3.11/psutil/tests
      copying psutil/tests/test_unicode.py -> build/lib.linux-aarch64-3.11/psutil/tests
      copying psutil/tests/test_contracts.py -> build/lib.linux-aarch64-3.11/psutil/tests
      copying psutil/tests/test_osx.py -> build/lib.linux-aarch64-3.11/psutil/tests
      copying psutil/tests/test_aix.py -> build/lib.linux-aarch64-3.11/psutil/tests
      copying psutil/tests/test_linux.py -> build/lib.linux-aarch64-3.11/psutil/tests
      copying psutil/tests/runner.py -> build/lib.linux-aarch64-3.11/psutil/tests
      copying psutil/tests/test_connections.py -> build/lib.linux-aarch64-3.11/psutil/tests
      copying psutil/tests/test_testutils.py -> build/lib.linux-aarch64-3.11/psutil/tests
      running build_ext
      creating build/temp.linux-aarch64-3.11
      creating build/temp.linux-aarch64-3.11/psutil
      aarch64-linux-gnu-gcc -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -DPSUTIL_POSIX=1 -DPSUTIL_SIZEOF_PID_T=4 -DPSUTIL_VERSION=595 -DPy_LIMITED_API=0x03060000 -DPSUTIL_LINUX=1 -I/usr/include/python3.11 -c psutil/_psutil_common.c -o build/temp.linux-aarch64-3.11/psutil/_psutil_common.o
      aarch64-linux-gnu-gcc -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -DPSUTIL_POSIX=1 -DPSUTIL_SIZEOF_PID_T=4 -DPSUTIL_VERSION=595 -DPy_LIMITED_API=0x03060000 -DPSUTIL_LINUX=1 -I/usr/include/python3.11 -c psutil/_psutil_linux.c -o build/temp.linux-aarch64-3.11/psutil/_psutil_linux.o
      aarch64-linux-gnu-gcc -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -DPSUTIL_POSIX=1 -DPSUTIL_SIZEOF_PID_T=4 -DPSUTIL_VERSION=595 -DPy_LIMITED_API=0x03060000 -DPSUTIL_LINUX=1 -I/usr/include/python3.11 -c psutil/_psutil_posix.c -o build/temp.linux-aarch64-3.11/psutil/_psutil_posix.o
      aarch64-linux-gnu-gcc -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -g -fwrapv -O2 build/temp.linux-aarch64-3.11/psutil/_psutil_common.o build/temp.linux-aarch64-3.11/psutil/_psutil_linux.o build/temp.linux-aarch64-3.11/psutil/_psutil_posix.o -L/usr/lib/aarch64-linux-gnu -o build/lib.linux-aarch64-3.11/psutil/_psutil_linux.abi3.so
      aarch64-linux-gnu-gcc -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -DPSUTIL_POSIX=1 -DPSUTIL_SIZEOF_PID_T=4 -DPSUTIL_VERSION=595 -DPy_LIMITED_API=0x03060000 -DPSUTIL_LINUX=1 -I/usr/include/python3.11 -c psutil/_psutil_common.c -o build/temp.linux-aarch64-3.11/psutil/_psutil_common.o
      aarch64-linux-gnu-gcc -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -DPSUTIL_POSIX=1 -DPSUTIL_SIZEOF_PID_T=4 -DPSUTIL_VERSION=595 -DPy_LIMITED_API=0x03060000 -DPSUTIL_LINUX=1 -I/usr/include/python3.11 -c psutil/_psutil_posix.c -o build/temp.linux-aarch64-3.11/psutil/_psutil_posix.o
      aarch64-linux-gnu-gcc -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -g -fwrapv -O2 build/temp.linux-aarch64-3.11/psutil/_psutil_common.o build/temp.linux-aarch64-3.11/psutil/_psutil_posix.o -L/usr/lib/aarch64-linux-gnu -o build/lib.linux-aarch64-3.11/psutil/_psutil_posix.abi3.so
      running install
      running install_lib
      psutil could not be installed from sources. Perhaps Python header files are not installed. Try running:
        sudo apt-get install gcc python3-dev
      Traceback (most recent call last):
        File "/usr/lib/python3/dist-packages/pip/_vendor/pep517/in_process/_in_process.py", line 363, in <module>
          main()
        File "/usr/lib/python3/dist-packages/pip/_vendor/pep517/in_process/_in_process.py", line 345, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/usr/lib/python3/dist-packages/pip/_vendor/pep517/in_process/_in_process.py", line 261, in build_wheel
          return _build_backend().build_wheel(wheel_directory, config_settings,
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/usr/lib/python3/dist-packages/setuptools/build_meta.py", line 230, in build_wheel
          return self._build_with_temp_dir(['bdist_wheel'], '.whl',
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/usr/lib/python3/dist-packages/setuptools/build_meta.py", line 215, in _build_with_temp_dir
          self.run_setup()
        File "/usr/lib/python3/dist-packages/setuptools/build_meta.py", line 158, in run_setup
          exec(compile(code, __file__, 'exec'), locals())
        File "setup.py", line 497, in <module>
          main()
        File "setup.py", line 462, in main
          setup(**kwargs)
        File "/usr/lib/python3/dist-packages/setuptools/__init__.py", line 153, in setup
          return distutils.core.setup(**attrs)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/usr/lib/python3/dist-packages/setuptools/_distutils/core.py", line 148, in setup
          return run_commands(dist)
                 ^^^^^^^^^^^^^^^^^^
        File "/usr/lib/python3/dist-packages/setuptools/_distutils/core.py", line 163, in run_commands
          dist.run_commands()
        File "/usr/lib/python3/dist-packages/setuptools/_distutils/dist.py", line 967, in run_commands
          self.run_command(cmd)
        File "/usr/lib/python3/dist-packages/setuptools/_distutils/dist.py", line 986, in run_command
          cmd_obj.run()
        File "/usr/lib/python3/dist-packages/wheel/bdist_wheel.py", line 335, in run
          self.run_command('install')
        File "/usr/lib/python3/dist-packages/setuptools/_distutils/cmd.py", line 313, in run_command
          self.distribution.run_command(command)
        File "/usr/lib/python3/dist-packages/setuptools/_distutils/dist.py", line 986, in run_command
          cmd_obj.run()
        File "/usr/lib/python3/dist-packages/setuptools/command/install.py", line 68, in run
          return orig.install.run(self)
                 ^^^^^^^^^^^^^^^^^^^^^^
        File "/usr/lib/python3/dist-packages/setuptools/_distutils/command/install.py", line 622, in run
          self.run_command(cmd_name)
        File "/usr/lib/python3/dist-packages/setuptools/_distutils/cmd.py", line 313, in run_command
          self.distribution.run_command(command)
        File "/usr/lib/python3/dist-packages/setuptools/_distutils/dist.py", line 985, in run_command
          cmd_obj.ensure_finalized()
        File "/usr/lib/python3/dist-packages/setuptools/_distutils/cmd.py", line 107, in ensure_finalized
          self.finalize_options()
        File "/usr/lib/python3/dist-packages/setuptools/command/install_lib.py", line 17, in finalize_options
          self.set_undefined_options('install',('install_layout','install_layout'))
        File "/usr/lib/python3/dist-packages/setuptools/_distutils/cmd.py", line 290, in set_undefined_options
          setattr(self, dst_option, getattr(src_cmd_obj, src_option))
                                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/usr/lib/python3/dist-packages/setuptools/_distutils/cmd.py", line 103, in __getattr__
          raise AttributeError(attr)
      AttributeError: install_layout. Did you mean: 'install_platlib'?
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for psutil
Failed to build psutil
ERROR: Could not build wheels for psutil, which is required to install pyproject.toml-based projects
root@db974e421c12:/workspaces/my-ai-docker-image# pip3 show psutil                     
WARNING: Package(s) not found: psutil

How can I successfully install psutil?

Side note: If I change my Python version to 3.10.x, psutil installs just fine.

BillRaymond commented 10 months ago

Sorry, Ubuntu 22.04

ben9923 commented 10 months ago

Ubuntu is using glibc, no need to install musl-dev.

Notice you're running python3 -m pip install --upgrade pip before running update-alternatives. This means you upgraded pip to the latest 23.2.1 for Python 3.10, but for Python 3.11 the apt-provided pip 22.0.2 is still used.

Run update-alternatives before updating pip to actually have an up-to-date pip with proper 3.11 support - it should resolve the problem for you :)

BillRaymond commented 10 months ago

Thank you @ben9923! I was so focused on getting the correct version of Python3 installed that I had yet to consider that pip would be the problem. I just wanted to let you know that you fixed my situation, and all is functioning as expected. Thanks again!