Xilinx / PYNQ

Python Productivity for ZYNQ
http://www.pynq.io/
BSD 3-Clause "New" or "Revised" License
1.99k stars 817 forks source link

failure to upgrade pynq in virtual python environment #1432

Closed xlla closed 6 months ago

xlla commented 1 year ago

I am follow the instruction in section Python Source Code to upgrade my pynq from v3.0.0 to latest, but it failed due to the virtual python environment.

round 1, it can't find python module pynqutils inside sudo

ModuleNotFoundError: No module named 'pynqutils'
sudo pip3 install --upgrade --upgrade-strategy only-if-needed pynq
Requirement already satisfied: pynq in /usr/local/share/pynq-venv/lib/python3.10/site-packages (3.0.0)
Collecting pynq
  Downloading pynq-3.0.1.tar.gz (63.0 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 63.0/63.0 MB 836.7 kB/s eta 0:00:00
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: pynqmetadata>=0.0.1 in /usr/local/share/pynq-venv/lib/python3.10/site-packages (from pynq) (0.3.1)
Requirement already satisfied: pynqutils>=0.0.1 in /usr/local/share/pynq-venv/lib/python3.10/site-packages (from pynq) (0.1.0)

Building wheels for collected packages: pynq
  Building wheel for pynq (pyproject.toml) ... error
  error: subprocess-exited-with-error

  × Building wheel for pynq (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [2854 lines of output]

....
thousands copying 
....

      Traceback (most recent call last):
        File "/usr/local/share/pynq-venv/lib/python3.10/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 363, in 
          main()
        File "/usr/local/share/pynq-venv/lib/python3.10/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 345, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
        File "/usr/local/share/pynq-venv/lib/python3.10/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 261, in build_wheel
          return _build_backend().build_wheel(wheel_directory, config_settings,
        File "/usr/lib/python3/dist-packages/setuptools/build_meta.py", line 230, in build_wheel
          return self._build_with_temp_dir(['bdist_wheel'], '.whl',
        File "/usr/lib/python3/dist-packages/setuptools/build_meta.py", line 215, in _build_with_temp_dir
          self.run_setup()
        File "/usr/lib/python3/dist-packages/setuptools/build_meta.py", line 267, in run_setup
          super(_BuildMetaLegacyBackend,
        File "/usr/lib/python3/dist-packages/setuptools/build_meta.py", line 158, in run_setup
          exec(compile(code, __file__, 'exec'), locals())
        File "setup.py", line 431, in 
          setup(
        File "/usr/lib/python3/dist-packages/setuptools/__init__.py", line 153, in setup
          return distutils.core.setup(**attrs)
        File "/usr/lib/python3/dist-packages/setuptools/_distutils/core.py", line 148, in setup
          return run_commands(dist)
        File "/usr/lib/python3/dist-packages/setuptools/_distutils/core.py", line 163, in run_commands
          dist.run_commands()
        File "/usr/lib/python3/dist-packages/setuptools/_distutils/dist.py", line 967, in run_commands
          self.run_command(cmd)
        File "/usr/lib/python3/dist-packages/setuptools/_distutils/dist.py", line 986, in run_command
          cmd_obj.run()
        File "/usr/lib/python3/dist-packages/wheel/bdist_wheel.py", line 299, in run
          self.run_command('build')
        File "/usr/lib/python3/dist-packages/setuptools/_distutils/cmd.py", line 313, in run_command
          self.distribution.run_command(command)
        File "/usr/lib/python3/dist-packages/setuptools/_distutils/dist.py", line 986, in run_command
          cmd_obj.run()
        File "/usr/lib/python3/dist-packages/setuptools/_distutils/command/build.py", line 135, in run
          self.run_command(cmd_name)
        File "/usr/lib/python3/dist-packages/setuptools/_distutils/cmd.py", line 313, in run_command
          self.distribution.run_command(command)
        File "/usr/lib/python3/dist-packages/setuptools/_distutils/dist.py", line 986, in run_command
          cmd_obj.run()
        File "setup.py", line 367, in run
          copy_notebooks()
        File "setup.py", line 326, in copy_notebooks
          copy_overlay_notebooks(notebooks_dir, board)
        File "setup.py", line 202, in copy_overlay_notebooks
          from pynqutils.setup_utils import download_overlays
      ModuleNotFoundError: No module named 'pynqutils'
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for pynq
Failed to build pynq
ERROR: Could not build wheels for pynq, which is required to install pyproject.toml-based projects

account xilinx can import that module

xilinx@pynq:~$ python
Python 3.10.4 (main, Apr  2 2022, 09:04:19) [GCC 11.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from pynqutils.setup_utils import download_overlays
>>> quit()

sudo cannot import that module

xilinx@pynq:~$ sudo python3
Python 3.10.4 (main, Apr  2 2022, 09:04:19) [GCC 11.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from pynqutils.setup_utils import download_overlays
Traceback (most recent call last):
  File "", line 1, in 
ModuleNotFoundError: No module named 'pynqutils'
>>> quit()

round 2, execute as normal account, failure again

ERROR: Will not install to the user site because it will lack sys.path precedence to pynq in /usr/local/share/pynq-venv/lib/python3.10/site-packages
xilinx@pynq:~$ pip3 --proxy=http://10.0.0.17:10809 install --upgrade --upgrade-strategy only-if-needed pynq
Defaulting to user installation because normal site-packages is not writeable
Requirement already satisfied: pynq in /usr/local/share/pynq-venv/lib/python3.10/site-packages (3.0.0)
Collecting pynq
  Downloading pynq-3.0.1.tar.gz (63.0 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 63.0/63.0 MB 886.9 kB/s eta 0:00:00
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: typing-extensions>=3.7.4.3 in /usr/local/share/pynq-venv/lib/python3.10/site-packages (from pydantic->pynqmetadata>=0.0.1->pynq) (4.3.0)
ERROR: Will not install to the user site because it will lack sys.path precedence to pynq in /usr/local/share/pynq-venv/lib/python3.10/site-packages

round 3, use "sudo -E" option to keep current environment variables to sudo

xilinx@pynq:~$ sudo python3 -c "from pynqutils.setup_utils import download_overlays"
Traceback (most recent call last):
  File "", line 1, in 
ModuleNotFoundError: No module named 'pynqutils'

xilinx@pynq:~$ sudo -E python3 -c "from pynqutils.setup_utils import download_overlays"
xilinx@pynq:~$ 
ModuleNotFoundError: No module named 'pynqutils'
xilinx@pynq:~$ sudo -E pip3  install --upgrade --upgrade-strategy only-if-needed pynq
WARNING: The directory '/home/xilinx/.cache/pip' or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you should use sudo's -H flag.
Requirement already satisfied: pynq in /usr/local/share/pynq-venv/lib/python3.10/site-packages (3.0.0)
Collecting pynq
  Downloading pynq-3.0.1.tar.gz (63.0 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 63.0/63.0 MB 5.0 MB/s eta 0:00:00
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: pynq
  Building wheel for pynq (pyproject.toml) ... error
  error: subprocess-exited-with-error

  × Building wheel for pynq (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [2854 lines of output]
....
thousands copying 
....

        File "setup.py", line 202, in copy_overlay_notebooks
          from pynqutils.setup_utils import download_overlays
      ModuleNotFoundError: No module named 'pynqutils'
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for pynq
Failed to build pynq
ERROR: Could not build wheels for pynq, which is required to install pyproject.toml-based projects

round 4, use "sudo -E -H" option to keep current environment variables and change the $Home environment variable, it remove the warnings in "round 3", but still failure.

ModuleNotFoundError: No module named 'pynqutils'
xilinx@pynq:/home$ sudo -E -H  /usr/local/share/pynq-venv/bin/pip3 install --upgrade --upgrade-strategy only-if-needed pynq
Requirement already satisfied: pynq in /usr/local/share/pynq-venv/lib/python3.10/site-packages (3.0.0)
Collecting pynq
  Using cached pynq-3.0.1.tar.gz (63.0 MB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done

....

        File "setup.py", line 202, in copy_overlay_notebooks
          from pynqutils.setup_utils import download_overlays
      ModuleNotFoundError: No module named 'pynqutils'
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for pynq
Failed to build pynq
ERROR: Could not build wheels for pynq, which is required to install pyproject.toml-based projects

round 5, use "sudo -H -E env PATH=$PATH " option to keep more environment variables, keep failure

ModuleNotFoundError: No module named 'pynqutils'
xilinx@pynq:/home$ sudo -H -E env PATH=$PATH pip3 install --upgrade --upgrade-strategy only-if-needed pynq
Requirement already satisfied: pynq in /usr/local/share/pynq-venv/lib/python3.10/site-packages (3.0.0)
Collecting pynq
  Using cached pynq-3.0.1.tar.gz (63.0 MB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done

....

        File "setup.py", line 202, in copy_overlay_notebooks
          from pynqutils.setup_utils import download_overlays
      ModuleNotFoundError: No module named 'pynqutils'
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for pynq
Failed to build pynq
ERROR: Could not build wheels for pynq, which is required to install pyproject.toml-based projects

round 6, use 'sudo sh -c ". /usr/local/share/pynq-venv/bin/activate ;" ' to activate python virtual env direct inside sudo, just failure

ModuleNotFoundError: No module named 'pynqutils'
xilinx@pynq:/home$ sudo sh -c ". /usr/local/share/pynq-venv/bin/activate ; pip3 install --upgrade --upgrade-strategy only-if-needed pynq"
Requirement already satisfied: pynq in /usr/local/share/pynq-venv/lib/python3.10/site-packages (3.0.0)
Collecting pynq
  Using cached pynq-3.0.1.tar.gz (63.0 MB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done

....

        File "setup.py", line 202, in copy_overlay_notebooks
          from pynqutils.setup_utils import download_overlays
      ModuleNotFoundError: No module named 'pynqutils'
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for pynq
Failed to build pynq
ERROR: Could not build wheels for pynq, which is required to install pyproject.toml-based projects

round 7, add "source /etc/profile.d/pynq_venv.sh" to root's .bashrc, execute command by root, failure ~ failure !!!

ModuleNotFoundError: No module named 'pynqutils'
xilinx@pynq:/home$ sudo vi /root/.bashrc
xilinx@pynq:/home$ sudo -i
(pynq-venv) root@pynq:~# which python
/usr/local/share/pynq-venv/bin/python
(pynq-venv) root@pynq:~# cd /home/xilinx
(pynq-venv) root@pynq:/home/xilinx# pip3 install --upgrade --upgrade-strategy only-if-needed pynq
Requirement already satisfied: pynq in /usr/local/share/pynq-venv/lib/python3.10/site-packages (3.0.0)
Collecting pynq
  Using cached pynq-3.0.1.tar.gz (63.0 MB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done

....

        File "setup.py", line 202, in copy_overlay_notebooks
          from pynqutils.setup_utils import download_overlays
      ModuleNotFoundError: No module named 'pynqutils'
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for pynq
Failed to build pynq
ERROR: Could not build wheels for pynq, which is required to install pyproject.toml-based projects

Because slow arm speed and slow SD card write speed, I spend hours to struggle with this issue, it seems in some files, it will invoke a subprocess and execute command using default python instead of pyqn_venv one.

STFleming commented 6 months ago

Thanks for flagging this issue. Can you please try and install the pynqutils and pynqmetadata packages before upgrading:

source /etc/profile.d/pynq_venv.sh
python3 -m pip install pynqmetadata
python3 -m pip install pynqutils

If you are still having issues please feel free to post on discuss.pynq.io