search

pez-globo / pufferfish-software

All software for the Pufferfish ventilator.
Apache License 2.0
0 stars 1 forks source link

deploy/scripts/pyenv.sh completes (incorrectly) if commands fail #221

Closed ethanjli closed 4 years ago

ethanjli commented 4 years ago

Describe the bug The pyenv setup script does not set up pyenv correctly, and the backend cannot be started from the command-line. What seems to happen is that apt-get installation fails for some packages needed to build python (such as libffi-dev) when started from the script, which prevents python3 from being built by pyenv. Over the past hour or two I have been consistently reproducing this issue.

To Reproduce Steps to reproduce the behavior:

  1. Flash pufferfish.img onto an SD card
  2. Clone the repo
  3. Run deploy/install.sh. A few apt-get packages fail to be installed, as seen in the logs at the end of this bug report.
  4. Allow the script to restart the RPi.
  5. In pufferfish-software/backend, try to runpoetry run python3 -m ventserver.application. The following error message appears:
Python 2.7 will no longer be supported in the next feature release of Poetry (1.2).
You should consider updating your Python version to a supported one.

Note that you will still be able to manage Python 2.7 projects by using the env command.
See https://python-poetry.org/docs/managing-environments/ for more information.

The currently activated Python version 2.7.16 is not supported by the project (^3.7).
Trying to find and use a compatible version. 
Using python3 (3.7.3)
Skipping virtualenv creation, as specified in config file.
Traceback (most recent call last):
  File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/pi/pufferfish-software/backend/ventserver/application.py", line 7, in <module>
    import trio
ModuleNotFoundError: No module named 'trio'
  1. Try to run poetry install. The following error message appears:
Python 2.7 will no longer be supported in the next feature release of Poetry (1.2).
You should consider updating your Python version to a supported one.

Note that you will still be able to manage Python 2.7 projects by using the env command.
See https://python-poetry.org/docs/managing-environments/ for more information.

The currently activated Python version 2.7.16 is not supported by the project (^3.7).
Trying to find and use a compatible version. 
Using python3 (3.7.3)
Skipping virtualenv creation, as specified in config file.
Installing dependencies from lock file
Warning: The lock file is not up to date with the latest changes in pyproject.toml. You may be getting outdated dependencies. Run update to update them.

SolverProblemError

The current project's Python requirement (2.7.16) is not compatible with some of the required packages Python requirement:
  - pytest-trio requires Python >=3.5, so it will not be satisfied for Python 2.7.16

Because pytest-trio (0.6.0) requires Python >=3.5
 and no versions of pytest-trio match >0.6.0,<0.7.0, pytest-trio is forbidden.
So, because ventserver depends on pytest-trio (^0.6.0), version solving failed.
  1. Manually run the three apt-get install commands from deploy/scripts/pyenv.sh. A few new packages will be installed this time.
  2. Rundeploy/install.sh again. Then the script successfully completes without errors.

Expected behavior The installation should succeed; if the apt-get packages cannot be installed or the pyenv commands fail, the install script should error out instead of reporting completion and proceeding to the next steps.

Device:

Software:

Additional context Here are the logs from the first run of the install script:

********** Installing pyenv **********

Get:1 http://archive.raspberrypi.org/debian buster InRelease [32.6 kB]
Get:2 http://raspbian.raspberrypi.org/raspbian buster InRelease [15.0 kB]
Get:3 http://raspbian.raspberrypi.org/raspbian buster/main armhf Packages [13.0 MB]
Get:4 http://archive.raspberrypi.org/debian buster/main armhf Packages [331 kB]
Fetched 13.4 MB in 9s (1,410 kB/s)                                                                                                                           
Reading package lists... Done
Building dependency tree       
Reading state information... Done
33 packages can be upgraded. Run 'apt list --upgradable' to see them.
Reading package lists... Done
Building dependency tree       
Reading state information... Done
curl is already the newest version (7.64.0-4+deb10u1).
gcc is already the newest version (4:8.3.0-1+rpi2).
gcc set to manually installed.
make is already the newest version (4.2.1-1.2).
make set to manually installed.
wget is already the newest version (1.20.1-1.1).
wget set to manually installed.
zlib1g-dev is already the newest version (1:1.2.11.dfsg-1).
zlib1g-dev set to manually installed.
Suggested packages:
  sqlite3-doc
The following NEW packages will be installed:
  libffi-dev libsqlite3-dev
0 upgraded, 2 newly installed, 0 to remove and 33 not upgraded.
Need to get 846 kB of archives.
After this operation, 2,223 kB of additional disk space will be used.
Err:1 http://raspbian.raspberrypi.org/raspbian buster/main armhf libffi-dev armhf 3.2.1-9
  Temporary failure resolving 'raspbian.raspberrypi.org'
Err:2 http://raspbian.raspberrypi.org/raspbian buster/main armhf libsqlite3-dev armhf 3.27.2-3
  Temporary failure resolving 'raspbian.raspberrypi.org'
E: Failed to fetch http://raspbian.raspberrypi.org/raspbian/pool/main/libf/libffi/libffi-dev_3.2.1-9_armhf.deb  Temporary failure resolving 'raspbian.raspberrypi.org'
E: Failed to fetch http://raspbian.raspberrypi.org/raspbian/pool/main/s/sqlite3/libsqlite3-dev_3.27.2-3_armhf.deb  Temporary failure resolving 'raspbian.raspberrypi.org'
E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?
Reading package lists... Done
Building dependency tree       
Reading state information... Done
build-essential is already the newest version (12.6).
The following additional packages will be installed:
  bzip2-doc libncurses-dev
Suggested packages:
  ncurses-doc readline-doc libssl-doc
The following NEW packages will be installed:
  bzip2-doc libbz2-dev libncurses-dev libreadline-dev libssl-dev
0 upgraded, 5 newly installed, 0 to remove and 33 not upgraded.
Need to get 2,491 kB of archives.
After this operation, 8,820 kB of additional disk space will be used.
Err:1 http://raspbian.raspberrypi.org/raspbian buster/main armhf bzip2-doc all 1.0.6-9.2~deb10u1
  Temporary failure resolving 'raspbian.raspberrypi.org'
Err:2 http://archive.raspberrypi.org/debian buster/main armhf libssl-dev armhf 1.1.1d-0+deb10u3+rpt1
  Temporary failure resolving 'archive.raspberrypi.org'
Err:3 http://raspbian.raspberrypi.org/raspbian buster/main armhf libbz2-dev armhf 1.0.6-9.2~deb10u1
  Temporary failure resolving 'raspbian.raspberrypi.org'
Err:4 http://raspbian.raspberrypi.org/raspbian buster/main armhf libncurses-dev armhf 6.1+20181013-2+deb10u2
  Temporary failure resolving 'raspbian.raspberrypi.org'
Err:5 http://raspbian.raspberrypi.org/raspbian buster/main armhf libreadline-dev armhf 7.0-5
  Temporary failure resolving 'raspbian.raspberrypi.org'
E: Failed to fetch http://raspbian.raspberrypi.org/raspbian/pool/main/b/bzip2/bzip2-doc_1.0.6-9.2~deb10u1_all.deb  Temporary failure resolving 'raspbian.raspberrypi.org'
E: Failed to fetch http://raspbian.raspberrypi.org/raspbian/pool/main/b/bzip2/libbz2-dev_1.0.6-9.2~deb10u1_armhf.deb  Temporary failure resolving 'raspbian.raspberrypi.org'
E: Failed to fetch http://raspbian.raspberrypi.org/raspbian/pool/main/n/ncurses/libncurses-dev_6.1+20181013-2+deb10u2_armhf.deb  Temporary failure resolving 'raspbian.raspberrypi.org'
E: Failed to fetch http://raspbian.raspberrypi.org/raspbian/pool/main/r/readline/libreadline-dev_7.0-5_armhf.deb  Temporary failure resolving 'raspbian.raspberrypi.org'
E: Failed to fetch http://archive.raspberrypi.org/debian/pool/main/o/openssl/libssl-dev_1.1.1d-0+deb10u3+rpt1_armhf.deb  Temporary failure resolving 'archive.raspberrypi.org'
E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?
Reading package lists... Done
Building dependency tree       
Reading state information... Done
python-openssl is already the newest version (19.0.0-1).
python-openssl set to manually installed.
xz-utils is already the newest version (5.2.4-1).
xz-utils set to manually installed.
The following additional packages will be installed:
  libncurses-dev
Suggested packages:
  liblzma-doc ncurses-doc
The following NEW packages will be installed:
  libffi-dev liblzma-dev libncurses-dev libncurses5-dev libncursesw5-dev
0 upgraded, 5 newly installed, 0 to remove and 33 not upgraded.
Need to get 642 kB of archives.
After this operation, 2,592 kB of additional disk space will be used.
Err:1 http://raspbian.raspberrypi.org/raspbian buster/main armhf libffi-dev armhf 3.2.1-9
  Temporary failure resolving 'raspbian.raspberrypi.org'
Err:2 http://raspbian.raspberrypi.org/raspbian buster/main armhf liblzma-dev armhf 5.2.4-1
  Temporary failure resolving 'raspbian.raspberrypi.org'
Err:3 http://raspbian.raspberrypi.org/raspbian buster/main armhf libncurses-dev armhf 6.1+20181013-2+deb10u2
  Temporary failure resolving 'raspbian.raspberrypi.org'
Err:4 http://raspbian.raspberrypi.org/raspbian buster/main armhf libncurses5-dev armhf 6.1+20181013-2+deb10u2
  Temporary failure resolving 'raspbian.raspberrypi.org'
Err:5 http://raspbian.raspberrypi.org/raspbian buster/main armhf libncursesw5-dev armhf 6.1+20181013-2+deb10u2
  Temporary failure resolving 'raspbian.raspberrypi.org'
E: Failed to fetch http://raspbian.raspberrypi.org/raspbian/pool/main/libf/libffi/libffi-dev_3.2.1-9_armhf.deb  Temporary failure resolving 'raspbian.raspberrypi.org'
E: Failed to fetch http://raspbian.raspberrypi.org/raspbian/pool/main/x/xz-utils/liblzma-dev_5.2.4-1_armhf.deb  Temporary failure resolving 'raspbian.raspberrypi.org'
E: Failed to fetch http://raspbian.raspberrypi.org/raspbian/pool/main/n/ncurses/libncurses-dev_6.1+20181013-2+deb10u2_armhf.deb  Temporary failure resolving 'raspbian.raspberrypi.org'
E: Failed to fetch http://raspbian.raspberrypi.org/raspbian/pool/main/n/ncurses/libncurses5-dev_6.1+20181013-2+deb10u2_armhf.deb  Temporary failure resolving 'raspbian.raspberrypi.org'
E: Failed to fetch http://raspbian.raspberrypi.org/raspbian/pool/main/n/ncurses/libncursesw5-dev_6.1+20181013-2+deb10u2_armhf.deb  Temporary failure resolving 'raspbian.raspberrypi.org'
E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   285  100   285    0     0    823      0 --:--:-- --:--:-- --:--:--   826
Cloning into '/home/pi/.pyenv'...
remote: Enumerating objects: 714, done.
remote: Counting objects: 100% (714/714), done.
remote: Compressing objects: 100% (477/477), done.
remote: Total 714 (delta 372), reused 328 (delta 145), pack-reused 0
Receiving objects: 100% (714/714), 394.17 KiB | 3.40 MiB/s, done.
Resolving deltas: 100% (372/372), done.
Cloning into '/home/pi/.pyenv/plugins/pyenv-doctor'...
remote: Enumerating objects: 11, done.
remote: Counting objects: 100% (11/11), done.
remote: Compressing objects: 100% (9/9), done.
remote: Total 11 (delta 1), reused 2 (delta 0), pack-reused 0
Unpacking objects: 100% (11/11), done.
Cloning into '/home/pi/.pyenv/plugins/pyenv-installer'...
remote: Enumerating objects: 16, done.
remote: Counting objects: 100% (16/16), done.
remote: Compressing objects: 100% (13/13), done.
remote: Total 16 (delta 1), reused 8 (delta 0), pack-reused 0
Unpacking objects: 100% (16/16), done.
Cloning into '/home/pi/.pyenv/plugins/pyenv-update'...
remote: Enumerating objects: 10, done.
remote: Counting objects: 100% (10/10), done.
remote: Compressing objects: 100% (6/6), done.
Unpacking objects: 100% (10/10), done.
remote: Total 10 (delta 1), reused 6 (delta 0), pack-reused 0
Cloning into '/home/pi/.pyenv/plugins/pyenv-virtualenv'...
remote: Enumerating objects: 57, done.
remote: Counting objects: 100% (57/57), done.
remote: Compressing objects: 100% (51/51), done.
remote: Total 57 (delta 11), reused 20 (delta 0), pack-reused 0
Unpacking objects: 100% (57/57), done.
Cloning into '/home/pi/.pyenv/plugins/pyenv-which-ext'...
remote: Enumerating objects: 10, done.
remote: Counting objects: 100% (10/10), done.
remote: Compressing objects: 100% (6/6), done.
remote: Total 10 (delta 1), reused 6 (delta 0), pack-reused 0
Unpacking objects: 100% (10/10), done.

WARNING: seems you still have not added 'pyenv' to the load path.

# Load pyenv automatically by adding
# the following to ~/.bashrc:

export PATH="/home/pi/.pyenv/bin:$PATH"
eval "$(pyenv init -)"
eval "$(pyenv virtualenv-init -)"
Downloading Python-3.7.7.tar.xz...
-> https://www.python.org/ftp/python/3.7.7/Python-3.7.7.tar.xz
Installing Python-3.7.7...
WARNING: The Python bz2 extension was not compiled. Missing the bzip2 lib?
WARNING: The Python readline extension was not compiled. Missing the GNU readline lib?
ERROR: The Python ssl extension was not compiled. Missing the OpenSSL lib?

Please consult to the Wiki page to fix the problem.
https://github.com/pyenv/pyenv/wiki/Common-build-problems

BUILD FAILED (Raspbian 10 using python-build 1.2.21)

Inspect or clean up the working tree at /tmp/python-build.20201027121611.1544
Results logged to /tmp/python-build.20201027121611.1544.log

Last 10 log lines:
        install|*) ensurepip="" ;; \
    esac; \
     ./python -E -m ensurepip \
        $ensurepip --root=/ ; \
fi
Looking in links: /tmp/tmpn7xj901m
Collecting setuptools
Collecting pip
Installing collected packages: setuptools, pip
Successfully installed pip-19.2.3 setuptools-41.2.0
pyenv-virtualenv: `3.7.7' is not installed in pyenv.
pyenv: version `ventserver' not installed

Pyenv setup complete

********** Installing poetry **********

Retrieving Poetry metadata

# Welcome to Poetry!

This will download and install the latest version of Poetry,
a dependency and package manager for Python.

It will add the `poetry` command to Poetry's bin directory, located at:

$HOME/.poetry/bin

This path will then be added to your `PATH` environment variable by
modifying the profile file located at:

$HOME/.profile

You can uninstall at any time by executing this script with the --uninstall option,
and these changes will be reverted.

Installing version: 1.1.4
  - Downloading poetry-1.1.4-linux.tar.gz (57.03MB)

Poetry (1.1.4) is installed now. Great!

To get started you need Poetry's bin directory ($HOME/.poetry/bin) in your `PATH`
environment variable. Next time you log in this will be done
automatically.

To configure your current shell run `source $HOME/.poetry/env`

/home/pi/.poetry/lib/poetry/_vendor/py2.7/subprocess32.py:149: RuntimeWarning: The _posixsubprocess module is not being used. Child process reliability may suffer if your program uses threads.
  "program uses threads.", RuntimeWarning)

Python 2.7 will no longer be supported in the next feature release of Poetry (1.2).
You should consider updating your Python version to a supported one.

Note that you will still be able to manage Python 2.7 projects by using the env command.
See https://python-poetry.org/docs/managing-environments/ for more information.

/home/pi/pufferfish-software/deploy/scripts/poetry.sh: line 31: /home/pi/.pyenv/versions/3.7.7/envs/ventserver/bin/python: No such file or directory

Poetry setup complete