balena-io-examples / internetspeedtest

Log your internet download, upload and ping metrics ti influxDB, and Grafana it. Grafana all the things!
GNU General Public License v3.0
56 stars 18 forks source link

Raspberry pi zero support? #3

Closed dstewen closed 2 years ago

dstewen commented 3 years ago

I downloaded this about a week or 10 days ago. It was working with errors on my pi zero w.

Now getting the following errors when I try to restart the speedtest container via the Balena interface:

speedtest ============================================================================== speedtest
speedtest You may only use this Speedtest software and information generated speedtest from it for personal, non-commercial use, through a command line speedtest interface on a personal computer. Your use of this software is subject speedtest to the End User License Agreement, Terms of Use and Privacy Policy at speedtest these URLs: speedtest
speedtest https://www.speedtest.net/about/eula speedtest https://www.speedtest.net/about/terms speedtest https://www.speedtest.net/about/privacy speedtest
speedtest ============================================================================== speedtest
speedtest License acceptance recorded. Continuing. speedtest
speedtest Traceback (most recent call last): speedtest File "speedtest.py", line 34, in speedtest result = speedtest.test() speedtest File "speedtest.py", line 21, in test speedtest print ("Timestamp = " + str(result['timestamp'])) speedtest KeyError: 'timestamp' speedtest Traceback (most recent call last): speedtest File "speedtest.py", line 34, in speedtest result = speedtest.test() speedtest File "speedtest.py", line 21, in test speedtest print ("Timestamp = " + str(result['timestamp'])) speedtest KeyError: 'timestamp' speedtest [2020-12-05 01:29:39.598] [error] Configuration - Timeout was reached (TimeoutException) speedtest [2020-12-05 01:29:39.617] [error] Configuration - Cannot retrieve configuration document (0) speedtest [2020-12-05 01:29:39.627] [error] ConfigurationError - Could not retrieve or read configuration (Configuration) speedtest [2020-12-05 01:29:39.639] [error] ConfigurationError - Could not retrieve or read configuration (Configuration) speedtest {"type":"log","timestamp":"2020-12-05T01:29:39Z","message":"Configuration - Could not retrieve or read configuration (ConfigurationError)","level":"error"} speedtest Traceback (most recent call last): speedtest File "speedtest.py", line 34, in speedtest result = speedtest.test() speedtest File "speedtest.py", line 20, in test speedtest result = json.loads(response[0]) speedtest File "/usr/local/lib/python3.7/json/init.py", line 348, in loads speedtest return _default_decoder.decode(s) speedtest File "/usr/local/lib/python3.7/json/decoder.py", line 337, in decode speedtest obj, end = self.raw_decode(s, idx=_w(s, 0).end()) speedtest File "/usr/local/lib/python3.7/json/decoder.py", line 355, in raw_decode speedtest raise JSONDecodeError("Expecting value", s, err.value) from None speedtest json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0) speedtest [2020-12-05 01:29:39.598] [error] Configuration - Timeout was reached (TimeoutException) speedtest [2020-12-05 01:29:39.617] [error] Configuration - Cannot retrieve configuration document (0) speedtest [2020-12-05 01:29:39.627] [error] ConfigurationError - Could not retrieve or read configuration (Configuration) speedtest [2020-12-05 01:29:39.639] [error] ConfigurationError - Could not retrieve or read configuration (Configuration) speedtest {"type":"log","timestamp":"2020-12-05T01:29:39Z","message":"Configuration - Could not retrieve or read configuration (ConfigurationError)","level":"error"} speedtest Traceback (most recent call last): speedtest File "speedtest.py", line 34, in speedtest result = speedtest.test() speedtest File "speedtest.py", line 20, in test speedtest result = json.loads(response[0]) speedtest File "/usr/local/lib/python3.7/json/init.py", line 348, in loads speedtest return _default_decoder.decode(s) speedtest File "/usr/local/lib/python3.7/json/decoder.py", line 337, in decode speedtest obj, end = self.raw_decode(s, idx=_w(s, 0).end()) speedtest File "/usr/local/lib/python3.7/json/decoder.py", line 355, in raw_decode speedtest raise JSONDecodeError("Expecting value", s, err.value) from None speedtest json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0) speedtest Traceback (most recent call last): speedtest File "speedtest.py", line 34, in speedtest result = speedtest.test() speedtest File "speedtest.py", line 21, in test speedtest print ("Timestamp = " + str(result['timestamp'])) speedtest KeyError: 'timestamp' speedtest Traceback (most recent call last): speedtest File "speedtest.py", line 34, in speedtest result = speedtest.test() speedtest File "speedtest.py", line 21, in test speedtest print ("Timestamp = " + str(result['timestamp'])) speedtest KeyError: 'timestamp' speedtest Traceback (most recent call last): speedtest File "speedtest.py", line 34, in speedtest result = speedtest.test() speedtest File "speedtest.py", line 21, in test speedtest print ("Timestamp = " + str(result['timestamp'])) speedtest KeyError: 'timestamp' speedtest Traceback (most recent call last): speedtest File "speedtest.py", line 34, in speedtest result = speedtest.test() speedtest File "speedtest.py", line 21, in test speedtest print ("Timestamp = " + str(result['timestamp'])) speedtest KeyError: 'timestamp' speedtest [2020-12-05 01:31:38.818] [error] Configuration - Timeout was reached (TimeoutException) speedtest [2020-12-05 01:31:38.834] [error] Configuration - Cannot retrieve configuration document (0) speedtest [2020-12-05 01:31:38.845] [error] ConfigurationError - Could not retrieve or read configuration (Configuration) speedtest [2020-12-05 01:31:38.853] [error] ConfigurationError - Could not retrieve or read configuration (Configuration) speedtest {"type":"log","timestamp":"2020-12-05T01:31:38Z","message":"Configuration - Could not retrieve or read configuration (ConfigurationError)","level":"error"} speedtest Traceback (most recent call last): speedtest File "speedtest.py", line 34, in speedtest result = speedtest.test() speedtest File "speedtest.py", line 20, in test speedtest result = json.loads(response[0]) speedtest File "/usr/local/lib/python3.7/json/init.py", line 348, in loads speedtest return _default_decoder.decode(s) speedtest File "/usr/local/lib/python3.7/json/decoder.py", line 337, in decode speedtest obj, end = self.raw_decode(s, idx=_w(s, 0).end()) speedtest File "/usr/local/lib/python3.7/json/decoder.py", line 355, in raw_decode speedtest raise JSONDecodeError("Expecting value", s, err.value) from None speedtest json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0) speedtest [2020-12-05 01:31:38.818] [error] Configuration - Timeout was reached (TimeoutException) speedtest [2020-12-05 01:31:38.834] [error] Configuration - Cannot retrieve configuration document (0) speedtest [2020-12-05 01:31:38.845] [error] ConfigurationError - Could not retrieve or read configuration (Configuration) speedtest [2020-12-05 01:31:38.853] [error] ConfigurationError - Could not retrieve or read configuration (Configuration) speedtest {"type":"log","timestamp":"2020-12-05T01:31:38Z","message":"Configuration - Could not retrieve or read configuration (ConfigurationError)","level":"error"} speedtest Traceback (most recent call last): speedtest File "speedtest.py", line 34, in speedtest result = speedtest.test() speedtest File "speedtest.py", line 20, in test speedtest result = json.loads(response[0]) speedtest File "/usr/local/lib/python3.7/json/init.py", line 348, in loads speedtest return _default_decoder.decode(s) speedtest File "/usr/local/lib/python3.7/json/decoder.py", line 337, in decode speedtest obj, end = self.raw_decode(s, idx=_w(s, 0).end()) speedtest File "/usr/local/lib/python3.7/json/decoder.py", line 355, in raw_decode speedtest raise JSONDecodeError("Expecting value", s, err.value) from None speedtest json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0) speedtest [2020-12-05 01:32:24.706] [error] Configuration - Timeout was reached (TimeoutException) speedtest [2020-12-05 01:32:24.725] [error] Configuration - Cannot retrieve configuration document (0) speedtest [2020-12-05 01:32:24.736] [error] ConfigurationError - Could not retrieve or read configuration (Configuration)

Note. I found I had to change the speedtest Dockerfile to the following to be able to download an image that includes python


FROM balenalib/raspberry-pi-alpine-python:3.7.7-build

WORKDIR /app

RUN pip3 install requests influxdb 

RUN wget https://bintray.com/ookla/download/download_file?file_path=ookla-speedtest-1.0.0-arm-linux.tgz -O speedtest.tgz

RUN tar -xvf speedtest.tgz

COPY . /app

ENTRYPOINT [ "python", "-u", "speedtest.py" ]

So my first question would be if the above python is the correct one I should be using.

Second question I have is I have noticed big gaps in the Grafana display of the data. I've got FREQUENCY set to 1800 (30 minutes) with the SERVER_ID variable set to the 1 server I want to test against and if I restart the container I will normally get 1 or 2 runs of the speedtest data collection script before it stops collecting for me and I have to restart it again.

phil-d-wilson commented 3 years ago

@dstewen I've not tried this on a Pi Zero - there's a chance that the device does not have enough memory for the stack being used here. Can you check memory usage? My guess is that the OOM killer is killing the speedtest exe.

n1md4 commented 3 years ago

I am trying to install this in an RPI0 and the 'speedtest' never starts, it continues attempting to download, and has never gotten past more than 17%.

n1md4 commented 3 years ago

It doesn't appear memory is an issue:

root@4a68129:~# free -m
              total        used        free      shared  buff/cache   available
Mem:         490304      165420      150828       18136      174056      306924
Swap:        122572        6144      116428

I got some errors from the logs:

Failed to start service 'mqtt sha256:246cd51101f808074ee0d0aeeaf9d634d54fa9aa8cee59f83a9e3213b86f5b4a' due to 'Could not parse status code from docker error:  Error: socket hang up'
Failed to start service 'connector sha256:e5f06c9c72c47777851659d78654bdeff0aa77a6930f1c9bb5d86724eefb7a00' due to 'Could not parse status code from docker error:  Error: socket hang up'
Failed to start service 'influxdb sha256:0866da0fd5bb684b2b9b13002956f1d9806bc8eac6d62a98bf1056d4268a25c6' due to 'Could not parse status code from docker error:  Error: socket hang up'
Failed to download image 'registry2.balena-cloud.com/v2/12aa67f7e4e13d318d48bbfdd1ec392a@sha256:d8e0d14421283ae69f33524addacde945c09959b51399081e0bd49665b9c8ef7' due to 'connect ECONNREFUSED /var/run/balena-engine.sock'
Supervisor starting

At which point it restarts Downloading delta for image 'registry2.balena-cloud.com/v2/12aa67f7e4e13d318d48bbfdd1ec392a@sha256:d8e0d14421283ae69f33524addacde945c09959b51399081e0bd49665b9c8ef7'.

n1md4 commented 3 years ago

I'm guessing for my problem it is a lack of device space:

Filesystem                     Type      Size  Used Avail Use% Mounted on
devtmpfs                       devtmpfs  232M     0  232M   0% /dev
tmpfs                          tmpfs     240M  4.0K  240M   1% /tmp
/dev/disk/by-state/active      ext4      300M  243M   38M  87% /mnt/sysroot/active
/dev/disk/by-state/resin-state ext4       19M  224K   17M   2% /mnt/state
none                           aufs      300M  243M   38M  87% /
tmpfs                          tmpfs     240M  476K  239M   1% /dev/shm
tmpfs                          tmpfs     240M   25M  215M  11% /run
tmpfs                          tmpfs     240M     0  240M   0% /sys/fs/cgroup
tmpfs                          tmpfs     240M   20K  240M   1% /var/volatile
/dev/mmcblk0p1                 vfat       40M  8.3M   32M  21% /mnt/boot
/dev/mmcblk0p6                 ext4      1.2G  1.2G  1.9M 100% /mnt/data
n1md4 commented 3 years ago

It was not due to the 2G SD card. I added an 8G and the same problem exists. The only [filtered] debugging I can see is this:

26.02.21 16:05:30 (+0000) Downloading image 'registry2.balena-cloud.com/v2/12aa67f7e4e13d318d48bbfdd1ec392a@sha256:d8e0d14421283ae69f33524addacde945c09959b51399081e0bd49665b9c8ef7'
26.02.21 16:47:24 (+0000) Downloading delta for image 'registry2.balena-cloud.com/v2/12aa67f7e4e13d318d48bbfdd1ec392a@sha256:d8e0d14421283ae69f33524addacde945c09959b51399081e0bd49665b9c8ef7'
26.02.21 16:55:20 (+0000) Failed to download image 'registry2.balena-cloud.com/v2/12aa67f7e4e13d318d48bbfdd1ec392a@sha256:d8e0d14421283ae69f33524addacde945c09959b51399081e0bd49665b9c8ef7' due to '(HTTP code 404) no such image - no such image: registry2.balena-cloud.com/v2/12aa67f7e4e13d318d48bbfdd1ec392a:delta-beb9e543ffa7cc0e: No such image: registry2.balena-cloud.com/v2/12aa67f7e4e13d318d48bbfdd1ec392a:delta-beb9e543ffa7cc0e '
26.02.21 17:10:43 (+0000) Downloading delta for image 'registry2.balena-cloud.com/v2/12aa67f7e4e13d318d48bbfdd1ec392a@sha256:d8e0d14421283ae69f33524addacde945c09959b51399081e0bd49665b9c8ef7'
26.02.21 17:32:19 (+0000) Downloading delta for image 'registry2.balena-cloud.com/v2/12aa67f7e4e13d318d48bbfdd1ec392a@sha256:d8e0d14421283ae69f33524addacde945c09959b51399081e0bd49665b9c8ef7'
26.02.21 17:52:47 (+0000) Downloading delta for image 'registry2.balena-cloud.com/v2/12aa67f7e4e13d318d48bbfdd1ec392a@sha256:d8e0d14421283ae69f33524addacde945c09959b51399081e0bd49665b9c8ef7'
26.02.21 18:19:38 (+0000) Downloading delta for image 'registry2.balena-cloud.com/v2/12aa67f7e4e13d318d48bbfdd1ec392a@sha256:d8e0d14421283ae69f33524addacde945c09959b51399081e0bd49665b9c8ef7'

At which point, as noted above, the download starts over again.

n1md4 commented 3 years ago

Made some progress, though a number of variables changed at the same time sooo ... here are my observations / actions:

  1. github.com seemed to be up and down all day; either that or I have a dodgy connection, but everything else was working.
  2. cloned internetspeedtest repo, before pushing to an application -- not sure what difference that makes, if any.
  3. all containers now download images correctly, and from the cloud dash appear to be running, but ....

All services are in a constant state of restarting, no idea why:

 Restarting service 'dashboard sha256:8d36c94964aa50df8cac4aa56ca69becbd333fbab3f7320d20933a69773dfb72'
Service exited 'connector sha256:e5f06c9c72c47777851659d78654bdeff0aa77a6930f1c9bb5d86724eefb7a00'
Service exited 'dashboard sha256:8d36c94964aa50df8cac4aa56ca69becbd333fbab3f7320d20933a69773dfb72'
Restarting service 'speedtest sha256:3e2b380add093153b1e626be66e83e4d42c67c04a5ae1b982ac6ac7bbf4a252a'
Restarting service 'influxdb sha256:0866da0fd5bb684b2b9b13002956f1d9806bc8eac6d62a98bf1056d4268a25c6'
Service exited 'speedtest sha256:3e2b380add093153b1e626be66e83e4d42c67c04a5ae1b982ac6ac7bbf4a252a'
Restarting service 'connector sha256:e5f06c9c72c47777851659d78654bdeff0aa77a6930f1c9bb5d86724eefb7a00'
Restarting service 'dashboard sha256:8d36c94964aa50df8cac4aa56ca69becbd333fbab3f7320d20933a69773dfb72'
Service exited 'influxdb sha256:0866da0fd5bb684b2b9b13002956f1d9806bc8eac6d62a98bf1056d4268a25c6'
Restarting service 'influxdb sha256:0866da0fd5bb684b2b9b13002956f1d9806bc8eac6d62a98bf1056d4268a25c6'
Restarting service 'speedtest sha256:3e2b380add093153b1e626be66e83e4d42c67c04a5ae1b982ac6ac7bbf4a252a'
Service exited 'connector sha256:e5f06c9c72c47777851659d78654bdeff0aa77a6930f1c9bb5d86724eefb7a00'
Service exited 'dashboard sha256:8d36c94964aa50df8cac4aa56ca69becbd333fbab3f7320d20933a69773dfb72'
Service exited 'influxdb sha256:0866da0fd5bb684b2b9b13002956f1d9806bc8eac6d62a98bf1056d4268a25c6'
Service exited 'speedtest sha256:3e2b380add093153b1e626be66e83e4d42c67c04a5ae1b982ac6ac7bbf4a252a'

etc.

Any ideas?

n1md4 commented 3 years ago

Attempts to run 'device diagnostics' produces an error ssh client socket error while initiating SSH connection.

n1md4 commented 3 years ago

I get this from journalctl:

Feb 26 23:36:10 641ce56 balenad[1398]: time="2021-02-26T23:36:10.341271934Z" level=error msg="Handler for GET /containers/a6881aba394191dce214eb9a749a6c13060c2d93bbbb545aded9405a63707c7d/json returned error: write unix /var/run/balena-engine.sock->@: write: broken pipe"
Feb 26 23:36:10 641ce56 balenad[1398]: time="2021-02-26T23:36:10.351433934Z" level=error msg="Handler for GET /v1.40/containers/influxdb_3329913_1711990/json returned error: write unix /var/run/balena-engine.sock->@: write: broken pipe"

That log shows output for influxdb, but it happens for all services.

n1md4 commented 3 years ago

From diagnostics Supervisor is running, but may be unhealthy hmmm :thinking:

phil-d-wilson commented 3 years ago

Hey @n1md4 - that doesn't sound like a healthy device, rather than it being specific to running this repo. I'd pop along to the balena forums and get some support on the device itself.

I have been meaning to try and run this up on a PiZero, but haven't got round to it yet.

tmigone commented 3 years ago

Just to clarify here. The Dockerfiles on this project are all pinned to raspberrypi3 or arm32v6, neither will run on a Pi Zero. So as it stands Pi Zero devices are not supported.

jellyfish-bot commented 3 years ago

[tmigone] This issue has attached support thread https://jel.ly.fish/4ae32e2a-f6b3-4d54-a0d2-056335fe83b4

phil-d-wilson commented 2 years ago

With the release of the PiZero2, I don't personally have the time or energy to add support for the PiZeroV1. PRs welcome, but I won't be working on this myself.