search

balena-os / balenahup

BALENA Host os UPdater
https://balena.io/
36 stars 11 forks source link

HUP ultimately succeeds, but the dashboard reports errors to the user #342

Open pdcastro opened 3 years ago

pdcastro commented 3 years ago

A user attempted HUP 3 times on the same day (from balenaOS v2.71.5+rev1 to v2.73.1+rev1, and from supervisor v12.3.5 to v12.4.6), getting errors on each stage. Ultimately HUP succeeded, but the user was left with the impression that it didn't, because of the errors reported to them through the dashboard (screenshot attached to the linked support thread) and the HUP logs (also attached to the support thread).

Some of the errors:

1st attempt:

Apr 28 13:54:30 ad37efc update-resin-supervisor[11901]: Getting image name and tag...
Apr 28 13:54:30 ad37efc update-resin-supervisor[11901]: Supervisor configuration found from API.
Apr 28 13:54:30 ad37efc update-resin-supervisor[11901]: Getting image id...
Apr 28 13:54:31 ad37efc update-resin-supervisor[11916]: Error: No such object: registry2.balena-cloud.com/v2/ee8...:latest
Apr 28 13:54:31 ad37efc update-resin-supervisor[11901]: Pulling supervisor registry2.balena-cloud.com/v2/ee8...:latest...
Apr 28 13:54:46 ad37efc update-resin-supervisor[11933]: Error response from daemon: Get https://registry2.balena-cloud.com/v2/: 
net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
Apr 28 13:54:46 ad37efc systemd[1]: run-update-supervisor.service: Main process exited, code=exited, status=1/FAILURE
Apr 28 13:54:46 ad37efc systemd[1]: run-update-supervisor.service: Failed with result 'exit-code'.

2nd attempt:

Apr 28 13:54:30 ad37efc update-resin-supervisor[11901]: Getting image name and tag...
Apr 28 13:54:30 ad37efc update-resin-supervisor[11901]: Supervisor configuration found from API.
Apr 28 13:54:30 ad37efc update-resin-supervisor[11901]: Getting image id...
Apr 28 13:54:31 ad37efc update-resin-supervisor[11916]: Error: No such object: registry2.balena-cloud.com/v2/ee8...:latest
Apr 28 13:54:31 ad37efc update-resin-supervisor[11901]: Pulling supervisor registry2.balena-cloud.com/v2/ee8...:latest...
Apr 28 13:54:46 ad37efc update-resin-supervisor[11933]: Error response from daemon: Get https://registry2.balena-cloud.com/v2/: 
net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
Apr 28 13:54:46 ad37efc systemd[1]: run-update-supervisor.service: Main process exited, code=exited, status=1/FAILURE
Apr 28 13:54:46 ad37efc systemd[1]: run-update-supervisor.service: Failed with result 'exit-code'.

3rd attempt:

delta-21cc0a84748dc8f8: Pulling from v2/220...
4f4fb700ef54: Pulling fs layer
4f4fb700ef54: Pull complete
unexpected EOF. Invalid delta tar archive
[000000251][LOG]Image type delta, location 'registry2.balena-cloud.com/v2/220...:delta-21c...' failed or not found, trying another source
[000000251][LOG]Running pre-update fixes for raspberrypi4-64
Removing start_db.elf from boot partition
Removing fixup_db.dat from boot partition
[000000251][LOG]Starting hostapp-update
Error response from daemon: Get https://registry2.balena-cloud.com/v2/v2/220.../manifests/sha256:24d...: Get https://api.balena-cloud.com/auth/v1/token?account=d_ad3...&scope=repository%3Av2%2F220...%3Apull&service=registry2.balena-cloud.com:
net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
[000000266][LOG]Image type balena_registry, location 'registry2.balena-cloud.com/v2/220...@sha256:24d...'
failed or not found, trying another source
[000000266][ERROR]all hostapp-update attempts have failed...

(Some hashes were replaced with ellipsis above -- see fuller logs in the linked support thread.)

This issue might be substantially solved if the dashboard reported a successful update in case balenaOS and the supervisor were detected to be already running at their target versions.

jellyfish-bot commented 3 years ago

[pdcastro] This issue has attached support thread https://jel.ly.fish/2b6026b6-3b51-4f5a-858c-c3c88b6be671