Flashing PiSCSI SD images with Raspberry Pi Imager v1.8.1 breaks piscsi-web

[benjamink]

Something changed with the Raspberry Pi Imager between v1.7.4 & v1.8.1 that seems to affect network startup & the 1st run of the piscsi-web service (which runs piscsi/python/web/start.sh). The initial pip install fails leaving the bjoern package broken. Looking at the beginning of the piscsi-web log seems to indicate pip can't access the Internet right away:

-- Journal begins at Mon 2023-11-13 12:08:03 GMT, ends at Thu 2023-11-16 01:20:51 GMT. --
Nov 13 12:08:23 piscsi systemd[1]: Started PiSCSI-Web service.
Nov 13 12:08:26 piscsi PISCSIWEB[485]: Creating python venv for web server
Nov 13 12:08:40 piscsi PISCSIWEB[485]: Activating venv
Nov 13 12:08:40 piscsi PISCSIWEB[485]: Installing requirements.txt
Nov 13 12:08:47 piscsi PISCSIWEB[581]: Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Nov 13 12:08:47 piscsi PISCSIWEB[581]: WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection>
Nov 13 12:08:47 piscsi PISCSIWEB[581]: WARNING: Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection>
Nov 13 12:08:48 piscsi PISCSIWEB[581]: WARNING: Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection>
Nov 13 12:08:50 piscsi PISCSIWEB[581]: WARNING: Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection>
Nov 13 12:08:54 piscsi PISCSIWEB[581]: WARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection>
Nov 13 12:08:55 piscsi PISCSIWEB[581]: WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection>
Nov 13 12:08:55 piscsi PISCSIWEB[581]: WARNING: Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection>
Nov 13 12:08:56 piscsi PISCSIWEB[581]: WARNING: Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection>
Nov 13 12:08:58 piscsi PISCSIWEB[581]: WARNING: Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection>
Nov 13 12:09:02 piscsi PISCSIWEB[581]: WARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection>
Nov 13 12:09:02 piscsi PISCSIWEB[581]: ERROR: Could not find a version that satisfies the requirement wheel
Nov 13 12:09:02 piscsi PISCSIWEB[581]: ERROR: No matching distribution found for wheel
Nov 13 12:09:02 piscsi systemd[1]: piscsi-web.service: Main process exited, code=exited, status=1/FAILURE
Nov 13 12:09:02 piscsi systemd[1]: piscsi-web.service: Failed with result 'exit-code'.
Nov 13 12:09:02 piscsi systemd[1]: piscsi-web.service: Consumed 19.883s CPU time.
Nov 13 12:09:03 piscsi systemd[1]: piscsi-web.service: Scheduled restart job, restart counter is at 1.
Nov 13 12:09:03 piscsi systemd[1]: Stopped PiSCSI-Web service.

This page talks about how to properly wait for networking to actually be available as a prerequisite in Systemd init scripts:

https://www.freedesktop.org/wiki/Software/systemd/NetworkTarget/

Steps to fix appear to be:

1. Enable the proper systemd-networkd-wait-online.service is enabled

   systemd-networkd-wait-online.service

2. Add a Wants line to the piscsi-web.service init so it looks like this:

   [Unit]
   Description=PiSCSI-Web service
   After=network-online.target piscsi.service
   Wants=network-online.target piscsi.service
   
   [Service]
   Type=simple
   Restart=always
   ExecStart=/home/pi/piscsi/python/web/start.sh
   SyslogIdentifier=PISCSIWEB
   
   [Install]
   WantedBy=multi-user.target

[benjamink]

Another option is to make start.sh wait for pypi.org to be available before running the pip install. Something like this would work:

until ping -c1 -q pypi.org >/dev/null; do sleep 2; done;

[benjamink]

This will just do a DNS query rather than try to communicate with the remote site:

until getent hosts pypi.org >/dev/null; do sleep 2; done;

[cefiar]

Maybe something like:

for i in {1..30}; do
        sleep 3
        getent hosts pypi.org >/dev/null 2&>1 && break;
done
if [ $? -ne 0 ]; then
  echo "DNS resolution failure - pypi.org"
  ERROR=1
fi

This tries to resolve it 30 times with 3 second gaps between attempts, then exits with the above error.

Needs to go before line 33 (currently) where ERROR is checked.

Note: The sleep needs to occur before getent, else you'll be checking the exit value of the sleep command.

[cefiar]

Note: Exiting if there is no internet might not be the best idea though, as this stops people being able to use the web interface on a local network that has no internet connection. Maybe putting something similar in a function and calling it when net is absolutely needed and spitting out a msg when it can't resolve the name instead.

[cefiar]

A longer term way to fix this is proposed in #1153 , which aims to set up a proper pip local cache to avoid going to the net where possible.