search

tiny-pilot / ansible-role-tinypilot

DEPRECATED: Merged into core tiny-pilot/tinypilot repo
https://tinypilotkvm.com
MIT License
118 stars 37 forks source link

create TinyPilot virtualenv fails #28

Closed keithnet closed 4 years ago

keithnet commented 4 years ago

The last tasks seems to fail: TASK [mtlynch.tinypilot : create TinyPilot virtualenv] ***** fatal: [localhost]: FAILED! => {"changed": false, "cmd": "/opt/tinypilot/venv/bin/pip3 list --format=freeze", "msg": "[Errno 2] No such file or directory: b'/opt/tinypilot/venv/bin/pip3': b'/opt/tinypilot/venv/bin/pip3'", "rc": 2}

Full output: root@raspberrypi:/home/pi# curl -sS https://raw.githubusercontent.com/mtlynch/tinypilot/master/quick-install | bash -

PLAY [localhost] ***

TASK [Gathering Facts] ***** ok: [localhost]

TASK [mtlynch.ustreamer : collect universal required apt packages] ***** ok: [localhost]

TASK [mtlynch.ustreamer : collect Raspberry Pi OS specific required apt packages] *** ok: [localhost]

TASK [mtlynch.ustreamer : collect Debian-specific required apt packages] *** skipping: [localhost]

TASK [mtlynch.ustreamer : collect Ubuntu-specific required apt packages] *** skipping: [localhost]

TASK [mtlynch.ustreamer : install uStreamer pre-requisite packages] **** ok: [localhost]

TASK [mtlynch.ustreamer : create ustreamer group] ** ok: [localhost]

TASK [mtlynch.ustreamer : create ustreamer user] *** ok: [localhost]

TASK [mtlynch.ustreamer : create uStreamer folder] ***** ok: [localhost]

TASK [mtlynch.ustreamer : get uStreamer repo] ** ok: [localhost]

TASK [mtlynch.ustreamer : clean repository if needed] ** skipping: [localhost]

TASK [mtlynch.ustreamer : enable OpenMax IL acceleration on Pi OS] ***** ok: [localhost]

TASK [mtlynch.ustreamer : build uStreamer] ***** ok: [localhost]

TASK [mtlynch.ustreamer : fix uStreamer folder permissions] **** ok: [localhost]

TASK [mtlynch.ustreamer : install uStreamer as a service] ** ok: [localhost]

TASK [mtlynch.ustreamer : enable systemd uStreamer service file] *** ok: [localhost]

TASK [geerlingguy.nginx : Include OS-specific variables.] ** ok: [localhost]

TASK [geerlingguy.nginx : Define nginx_user.] ** ok: [localhost]

TASK [geerlingguy.nginx : include_tasks] *** skipping: [localhost]

TASK [geerlingguy.nginx : include_tasks] *** skipping: [localhost]

TASK [geerlingguy.nginx : include_tasks] *** included: /tmp/tmp.yQ9opggT79/geerlingguy.nginx/tasks/setup-Debian.yml for localhost

TASK [geerlingguy.nginx : Update apt cache.] *** ok: [localhost]

TASK [geerlingguy.nginx : Ensure nginx is installed.] ** ok: [localhost]

TASK [geerlingguy.nginx : include_tasks] *** skipping: [localhost]

TASK [geerlingguy.nginx : include_tasks] *** skipping: [localhost]

TASK [geerlingguy.nginx : include_tasks] *** skipping: [localhost]

TASK [geerlingguy.nginx : Remove default nginx vhost config file (if configured).] *** ok: [localhost]

TASK [geerlingguy.nginx : Ensure nginx_vhost_path exists.] ***** ok: [localhost]

TASK [geerlingguy.nginx : Add managed vhost config files.] ***** ok: [localhost] => (item={'listen': '80 default_server', 'server_name': 'tinypilot', 'root': '/opt/tinypilot', 'index': 'index.html', 'extra_parameters': 'proxy_buffers 16 16k;\nproxy_buffer_size 16k;\nproxy_set_header Host $host;\nproxy_set_header X-Forwarded-For $remote_addr;\nproxy_http_version 1.1;\n\nlocation /socket.io {\n proxy_pass http://tinypilot;\n proxy_set_header Upgrade $http_upgrade;\n proxy_set_header Connection "Upgrade";\n}\nlocation /state {\n proxy_pass http://ustreamer;\n}\nlocation /stream {\n proxy_pass http://ustreamer;\n}\nlocation / {\n proxy_pass http://tinypilot;\n}\nlocation ~* ^/.+\.(html|js|js.map|css|jpeg|png|ico)$ {\n root "/opt/tinypilot/app/static";\n}\n'})

TASK [geerlingguy.nginx : Remove managed vhost config files.] ** skipping: [localhost] => (item={'listen': '80 default_server', 'server_name': 'tinypilot', 'root': '/opt/tinypilot', 'index': 'index.html', 'extra_parameters': 'proxy_buffers 16 16k;\nproxy_buffer_size 16k;\nproxy_set_header Host $host;\nproxy_set_header X-Forwarded-For $remote_addr;\nproxy_http_version 1.1;\n\nlocation /socket.io {\n proxy_pass http://tinypilot;\n proxy_set_header Upgrade $http_upgrade;\n proxy_set_header Connection "Upgrade";\n}\nlocation /state {\n proxy_pass http://ustreamer;\n}\nlocation /stream {\n proxy_pass http://ustreamer;\n}\nlocation / {\n proxy_pass http://tinypilot;\n}\nlocation ~* ^/.+\.(html|js|js.map|css|jpeg|png|ico)$ {\n root "/opt/tinypilot/app/static";\n}\n'})

TASK [geerlingguy.nginx : Remove legacy vhosts.conf file.] ***** ok: [localhost]

TASK [geerlingguy.nginx : Copy nginx configuration in place.] ** ok: [localhost]

TASK [geerlingguy.nginx : Ensure nginx service is running as configured.] ** ok: [localhost]

TASK [mtlynch.tinypilot : enable dwc2 driver in boot config] *** ok: [localhost]

TASK [mtlynch.tinypilot : enable dwc2 driver in modules] *** ok: [localhost]

TASK [mtlynch.tinypilot : copy HID initializer script] ***** ok: [localhost]

TASK [mtlynch.tinypilot : install HID initializer as a service] **** ok: [localhost]

TASK [mtlynch.tinypilot : enable systemd HID initializer service file] ***** ok: [localhost]

TASK [mtlynch.tinypilot : install TinyPilot pre-requisite packages] **** ok: [localhost]

TASK [mtlynch.tinypilot : create tinypilot group] ** ok: [localhost]

TASK [mtlynch.tinypilot : create tinypilot user] *** ok: [localhost]

TASK [mtlynch.tinypilot : enable passwordless sudo for shutdown command] *** ok: [localhost]

TASK [mtlynch.tinypilot : create TinyPilot folder] ***** ok: [localhost]

TASK [mtlynch.tinypilot : get TinyPilot repo] ** ok: [localhost]

TASK [mtlynch.tinypilot : create TinyPilot virtualenv] ***** fatal: [localhost]: FAILED! => {"changed": false, "cmd": "/opt/tinypilot/venv/bin/pip3 list --format=freeze", "msg": "[Errno 2] No such file or directory: b'/opt/tinypilot/venv/bin/pip3': b'/opt/tinypilot/venv/bin/pip3'", "rc": 2}

PLAY RECAP ***** localhost : ok=35 changed=0 unreachable=0 failed=1 skipped=9 rescued=0 ignored=0

mtlynch commented 4 years ago

This looks similar to https://github.com/mtlynch/tinypilot/issues/85

Could you try upgrading your packages and then try again?

sudo apt-get update && \
  sudo apt-get upgrade -y
keithnet commented 4 years ago

Sorry for opening another issue.

My packages were up-to-date, but I tried again:

pi@raspberrypi:~ $ sudo apt-get update && \

sudo apt-get upgrade -y Hit:1 http://archive.raspberrypi.org/debian buster InRelease Hit:2 http://raspbian.raspberrypi.org/raspbian buster InRelease Reading package lists... Done Reading package lists... Done Building dependency tree Reading state information... Done Calculating upgrade... Done The following package was automatically installed and is no longer required: rpi-eeprom-images Use 'sudo apt autoremove' to remove it. 0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.

Re-ran quick-install. same error.

This is a brand new raspberry pie set up following all of your instructions just for this purpose.

Thx.

mtlynch commented 4 years ago

Thanks, @keithnet. Hmm, I'm trying to figure out why certain Pis run into this. Could you paste the output of this command?

ls -l /opt/tinypilot /opt/tinypilot/venv/ /opt/tinypilot/venv/bin/ && \
  cat /etc/os-release
keithnet commented 4 years ago

root@raspberrypi:/home/pi# ls -l /opt/tinypilot /opt/tinypilot/venv/ /opt/tinypilot/venv/bin/ && \

cat /etc/os-release /opt/tinypilot: total 2700 drwxr-xr-x 4 root root 4096 Aug 9 20:51 app -rw-r--r-- 1 root root 55 Aug 2 16:35 COPYRIGHT -rw-r--r-- 1 root root 2704070 Aug 2 16:35 demo.gif -rw-r--r-- 1 root root 71 Aug 2 16:35 dev_requirements.txt -rwxr-xr-x 1 root root 925 Aug 2 16:35 enable-usb-hid drwxr-xr-x 2 root root 4096 Aug 2 16:35 hooks -rw-r--r-- 1 root root 0 Aug 2 16:35 init.py -rw-r--r-- 1 root root 1053 Aug 2 16:35 LICENSE -rw-r--r-- 1 root root 65 Aug 2 16:35 MANIFEST.in -rwxr-xr-x 1 root root 856 Aug 9 20:51 quick-install -rw-r--r-- 1 root root 5470 Aug 9 20:51 README.md -rw-r--r-- 1 root root 70 Aug 2 16:35 requirements.txt -rwxr-xr-x 1 root root 877 Aug 2 16:35 run_build -rw-r--r-- 1 root root 465 Aug 2 16:35 setup.py drwxr-xr-x 6 root root 4096 Aug 2 16:35 venv

/opt/tinypilot/venv/: total 20 drwxr-xr-x 2 root root 4096 Aug 2 16:36 bin drwxr-xr-x 3 root root 4096 Aug 2 16:36 include drwxr-xr-x 3 root root 4096 Aug 2 16:35 lib -rw-r--r-- 1 root root 89 Aug 2 16:35 pyvenv.cfg drwxr-xr-x 3 root root 4096 Aug 2 16:35 share

/opt/tinypilot/venv/bin/: total 36 -rw-r--r-- 1 root root 2195 Aug 2 16:35 activate -rw-r--r-- 1 root root 1251 Aug 2 16:35 activate.csh -rw-r--r-- 1 root root 2403 Aug 2 16:35 activate.fish -rwxr-xr-x 1 root root 248 Aug 2 16:35 easy_install -rwxr-xr-x 1 root root 248 Aug 2 16:35 easy_install-3.7 -rwxr-xr-x 1 root root 226 Aug 2 16:36 flask -rwxr-xr-x 1 root root 230 Aug 2 16:35 pip -rwxr-xr-x 1 root root 230 Aug 2 16:35 pip3 -rwxr-xr-x 1 root root 230 Aug 2 16:35 pip3.7 lrwxrwxrwx 1 root root 7 Aug 2 16:35 python -> python3 lrwxrwxrwx 1 root root 36 Aug 2 16:35 python3 -> /tmp/tmp.Zg85Jt4Fyr/venv/bin/python3 PRETTY_NAME="Raspbian GNU/Linux 10 (buster)" NAME="Raspbian GNU/Linux" VERSION_ID="10" VERSION="10 (buster)" VERSION_CODENAME=buster ID=raspbian ID_LIKE=debian HOME_URL="http://www.raspbian.org/" SUPPORT_URL="http://www.raspbian.org/RaspbianForums" BUG_REPORT_URL="http://www.raspbian.org/RaspbianBugs"

mtlynch commented 4 years ago

That's strange because everything looks right. The file it's complaining about seems to be there:

fatal: [localhost]: FAILED! => {"changed": false, "cmd": "/opt/tinypilot/venv/bin/pip3 list --format=freeze", "msg": "[Errno 2] No such file or directory: b'/opt/tinypilot/venv/bin/pip3': b'/opt/tinypilot/venv/bin/pip3'", "rc": 2}

...

root@raspberrypi:/home/pi# ls -l /opt/tinypilot /opt/tinypilot/venv/ /opt/tinypilot/venv/bin/ && \ ... /opt/tinypilot/venv/bin/: total 36 -rwxr-xr-x 1 root root 230 Aug 2 16:35 pip3

I'll try to repro on my side and follow up.

mtlynch commented 4 years ago

I tried doing the exact same setup, and I'm not able to repro the failure. It installs successfully on my end, and I can't figure out what's different:

https://gist.github.com/mtlynch/653a0b287be1c29b03949cb95f494303

I'm going to keep investigating and see if I can find anything.

My only thought is maybe the fact that you're running as root? Can you try running the original quick-install command as the pi user?

keithnet commented 4 years ago

Just tried it as pi and not root and same result.

mtlynch commented 4 years ago

Okay, two other ideas.

  1. Can you try manually creating the virtualenv?
pushd /opt/tinypilot
python3 -m venv venv
. venv/bin/activate
pip install -r requirements.txt
deactivate
popd
  1. Can you try installing with this verbose version of the installer so we can try to get more hints?
curl -sS https://raw.githubusercontent.com/mtlynch/tinypilot/install-verbose/quick-install | bash -
keithnet commented 4 years ago

I tried the verbose version first. Here's the part where it died:

TASK [mtlynch.tinypilot : create TinyPilot virtualenv] ***** task path: /tmp/tmp.eZHTuUvE7U/mtlynch.tinypilot/tasks/main.yml:50

ESTABLISH LOCAL CONNECTION FOR USER: pi EXEC /bin/sh -c 'echo ~pi && sleep 0' EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /home/pi/.ansible/tmp `"&& mkdir /home/pi/.ansible/tmp/ansible-tmp-1597024575.7156525-6920-46199644149019 && echo ansible-tmp-1597024575.7156525-6920-46199644149019="` echo /home/pi/.ansible/tmp/ansible-tmp-1597024575.7156525-6920-46199644149019 `" ) && sleep 0' Using module file /tmp/tmp.eZHTuUvE7U/venv/lib/python3.7/site-packages/ansible/modules/packaging/language/pip.py PUT /home/pi/.ansible/tmp/ansible-local-566419teqpwn/tmp3uk6_p1r TO /home/pi/.ansible/tmp/ansible-tmp-1597024575.7156525-6920-46199644149019/AnsiballZ_pip.py EXEC /bin/sh -c 'chmod u+x /home/pi/.ansible/tmp/ansible-tmp-1597024575.7156525-6920-46199644149019/ /home/pi/.ansible/tmp/ansible-tmp-1597024575.7156525-6920-46199644149019/AnsiballZ_pip.py && sleep 0' EXEC /bin/sh -c 'sudo -H -S -n -u root /bin/sh -c '"'"'echo BECOME-SUCCESS-aglmhmkplstlhnrpflatbrihkhfnkozb ; /usr/bin/python3 /home/pi/.ansible/tmp/ansible-tmp-1597024575.7156525-6920-46199644149019/AnsiballZ_pip.py'"'"' && sleep 0' EXEC /bin/sh -c 'rm -f -r /home/pi/.ansible/tmp/ansible-tmp-1597024575.7156525-6920-46199644149019/ > /dev/null 2>&1 && sleep 0' The full traceback is: File "/tmp/ansible_pip_payload_o7jd6cyq/ansible_pip_payload.zip/ansible/module_utils/basic.py", line 2588, in run_command cmd = subprocess.Popen(args, **kwargs) File "/usr/lib/python3.7/subprocess.py", line 775, in __init__ restore_signals, start_new_session) File "/usr/lib/python3.7/subprocess.py", line 1522, in _execute_child raise child_exception_type(errno_num, err_msg, err_filename) fatal: [localhost]: FAILED! => { "changed": false, "cmd": "/opt/tinypilot/venv/bin/pip3 list --format=freeze", "invocation": { "module_args": { "chdir": null, "editable": false, "executable": null, "extra_args": null, "name": null, "requirements": "/opt/tinypilot/requirements.txt", "state": "present", "umask": null, "version": null, "virtualenv": "/opt/tinypilot/venv", "virtualenv_command": "python3 -m venv venv", "virtualenv_python": null, "virtualenv_site_packages": false } }, "msg": "[Errno 2] No such file or directory: b'/opt/tinypilot/venv/bin/pip3': b'/opt/tinypilot/venv/bin/pip3'", "rc": 2 } PLAY RECAP ********************************************************************* localhost : ok=35 changed=0 unreachable=0 failed=1 skipped=9 rescued=0 ignored=0 I'm going to hold off on manually creating the virtualenv until you tell me try that instead.
mtlynch commented 4 years ago

Yeah, it unfortunately doesn't give us much. Can you try the manual venv command?

keithnet commented 4 years ago

pi@raspberrypi:~ $ pushd /opt/tinypilot /opt/tinypilot ~ pi@raspberrypi:/opt/tinypilot $ python3 -m venv venv Error: [Errno 13] Permission denied: '/opt/tinypilot/venv/pyvenv.cfg' pi@raspberrypi:/opt/tinypilot $ . venv/bin/activate (venv) pi@raspberrypi:/opt/tinypilot $ pip install -r requirements.txt -bash: /opt/tinypilot/venv/bin/pip: /opt/tinypilot/venv/bin/python3: bad interpreter: No such file or directory (venv) pi@raspberrypi:/opt/tinypilot $ deactivate pi@raspberrypi:/opt/tinypilot $ popd ~

This help?

pi@raspberrypi:~ $ ls -l /opt/tinypilot/venv/bin/python3 lrwxrwxrwx 1 root root 36 Aug 2 16:35 /opt/tinypilot/venv/bin/python3 -> /tmp/tmp.Zg85Jt4Fyr/venv/bin/python3 pi@raspberrypi:~ $ ls -l /opt/tinypilot/venv/bin/python3 lrwxrwxrwx 1 root root 36 Aug 2 16:35 /opt/tinypilot/venv/bin/python3 -> /tmp/tmp.Zg85Jt4Fyr/venv/bin/python3 pi@raspberrypi:~ $ ls -l /opt/tinypilot/venv/bin/pip -rwxr-xr-x 1 root root 230 Aug 2 16:35 /opt/tinypilot/venv/bin/pip pi@raspberrypi:~ $ ls -l /tmp/tmp.Zg85Jt4Fyr/venv/bin/python3 ls: cannot access '/tmp/tmp.Zg85Jt4Fyr/venv/bin/python3': No such file or directory

mtlynch commented 4 years ago

This help?

Yes! Now I see what's going on. In the /opt/tinypilot virtualenv, /opt/tinypilot/venv/bin/python3 should point to the system-wide python3 binary at /usr/bin/python3, but it's pointing to the virtualenv copy in the temporary install directory. I think it's saying no such file because it doesn't have permissions to access the one in the install directory.

Let me think about how to fix this correctly. In the meantime, I think this should do as a workaround:

sudo rm -rf /opt/tinypilot/venv && \
  sudo chown $(whoami) /opt/tinypilot -R && \
  pushd /opt/tinypilot && \
  python3 -m venv venv && \
  . venv/bin/activate && \
  pip install -r requirements.txt && \
  deactivate && \
  popd

And then re-run the normal installer.

keithnet commented 4 years ago

Done. Normal installer worked without error.

I do get 502 Bad Gateway | nginx/1.14.2 when I visit the web server on my pi tho.

keithnet commented 4 years ago

I do get 502 Bad Gateway | nginx/1.14.2 when I visit the web server on my pi tho.

A reboot of the pi fixed it.

mtlynch commented 4 years ago

Thanks for your patience on this. I believe #32 fixes this.