search

efahl / owut

OpenWrt Upgrade Tool
13 stars 0 forks source link

no reboot/install after image validation #16

Open nextgen-networks opened 1 week ago

nextgen-networks commented 1 week ago

My system: Banana Pi BPI-R4 wih latest Snapshot

System does not reboot and install - even after successful verification of the created image and announcing a reboot (see shell output)

root@BananaPi4:~# owut install -v
owut - OpenWrt Upgrade Tool 2024.06.24~1b222bdc-r1 (/usr/bin/owut)
Verifying   : /tmp/firmware.bin (25690919 bytes) against /tmp/firmware.sha256sums
  Saved sha256 matches
  (null)
Checks complete, image is valid.
Installing /tmp/firmware.bin and rebooting...
root@BananaPi4:~#

Any ideas how to solve this?

Thanks for your great script and to anyone who improves and support it!

efahl commented 1 week ago

(null)

That's definitely a bug in owut, there should be a list of info from sysupgrade instead of (null):

Verifying   : /tmp/firmware.bin (33686815 bytes) against /tmp/firmware.sha256sums
  Saved sha256 matches
  Thu Jun 27 07:47:48 PDT 2024 upgrade: Image metadata not present
  Thu Jun 27 07:47:48 PDT 2024 upgrade: Reading partition table from bootdisk...
  Thu Jun 27 07:47:48 PDT 2024 upgrade: Extract boot sector from the image
  Thu Jun 27 07:47:48 PDT 2024 upgrade: Reading partition table from image...

Could you run the following and post the result of the final command line? I'm curious what sysupgrade output you get, and what its final status code (that $?) is...

owut download <whatever other parameters you used>
sysupgrade --test /tmp/firmware.bin ; echo $?
nextgen-networks commented 1 week ago

@efahl Thanks for your reply, here is the output of those two commands:

BusyBox v1.36.1 (2024-06-26 09:21:04 UTC) built-in shell (ash)

  _______                     ________        __
 |       |.-----.-----.-----.|  |  |  |.----.|  |_
 |   -   ||  _  |  -__|     ||  |  |  ||   _||   _|
 |_______||   __|_____|__|__||________||__|  |____|
          |__| W I R E L E S S   F R E E D O M
 -----------------------------------------------------
 OpenWrt SNAPSHOT, r26805-b43c0a915f
 -----------------------------------------------------
root@BananaPi4:~# owut download -v
owut - OpenWrt Upgrade Tool 2024.06.24~1b222bdc-r1 (/usr/bin/owut)
Downloaded https://sysupgrade.openwrt.org/json/v1/overview.json to /tmp/owut-overview.json
Downloaded https://mirror-03.infra.openwrt.org/.versions.json to /tmp/owut-versions.json
Downloaded https://sysupgrade.openwrt.org/json/v1/snapshots/targets/mediatek/filogic/bananapi_bpi-r4.json to /tmp/owut-platform.json
Downloaded https://sysupgrade.openwrt.org/json/v1/snapshots/packages/aarch64_cortex-a53-index.json to /tmp/owut-packages-arch.json
Downloaded https://sysupgrade.openwrt.org/json/v1/snapshots/targets/mediatek/filogic/index.json to /tmp/owut-packages-plat.json
Board-name     bananapi_bpi-r4
Target         mediatek/filogic
Root-FS-type   squashfs
Sys-type       sysupgrade
Package-arch   aarch64_cortex-a53
Version-from   SNAPSHOT r26805-b43c0a915f (kernel 6.6.35)
Version-to     SNAPSHOT r26809-7c9644a7b5 (kernel 6.6.35)
Build-FS-type  squashfs
Build-at       2024-06-27T15:59:31Z
Image-prefix   openwrt-mediatek-filogic-bananapi_bpi-r4
Image-file     openwrt-mediatek-filogic-bananapi_bpi-r4-squashfs-sysupgrade.itb
Image-URL      https://mirror-03.infra.openwrt.org/snapshots/targets/mediatek/filogic/openwrt-mediatek-filogic-bananapi_bpi-r4-squashfs-sysupgrade.itb
Installed      320 packages
Top-level       75 packages
Default         45 packages
User-installed  41 packages (top-level only)

Package version changes:
  base-files                        1606~b43c0a915f                            1606~7c9644a7b5
  luci-mod-network                  24.149.31337~7226e13                       24.179.53069~5755899
  luci-proto-wireguard              24.148.86114~fbbfdb4                       24.179.65998~74f366a
3 packages are out-of-date.

Default package analysis:
  Default              Provided-by
  nftables             not installed, possibly replaced by another package
  wpad-basic-mbedtls   not installed, possibly replaced by another package

Downloaded https://mirror-03.infra.openwrt.org/snapshots/faillogs/aarch64_cortex-a53/ to /tmp/owut-failures-feeds.html
There are currently package build failures for SNAPSHOT aarch64_cortex-a53:
  Feed: packages
Downloaded https://mirror-03.infra.openwrt.org/snapshots/faillogs/aarch64_cortex-a53/packages/ to /tmp/owut-failures-packages.html
    basicstation               Fri Jun 28 03:51:58 2024 - not installed
    elektra                    Fri Jun 28 05:07:05 2024 - not installed
    freeradius3                Fri Jun 28 05:39:18 2024 - not installed
    hwinfo                     Fri Jun 28 03:26:42 2024 - not installed
    kadnode                    Fri Jun 28 03:04:19 2024 - not installed
    libuhttpd                  Fri Jun 28 04:09:50 2024 - not installed
    micropython                Fri Jun 28 03:09:22 2024 - not installed
    schroot                    Fri Jun 28 05:27:25 2024 - not installed
    shadowsocks-libev          Fri Jun 28 04:18:20 2024 - not installed
    umurmur                    Fri Jun 28 04:22:04 2024 - not installed
  Feed: telephony
Downloaded https://mirror-03.infra.openwrt.org/snapshots/faillogs/aarch64_cortex-a53/telephony/ to /tmp/owut-failures-telephony.html
    freeswitch                 Fri Jun 28 05:27:55 2024 - not installed
    freeswitch-mod-bcg729      Fri Jun 28 06:09:00 2024 - not installed
    freetdm                    Fri Jun 28 06:09:39 2024 - not installed
Failures don't affect this device, details at
  https://mirror-03.infra.openwrt.org/snapshots/faillogs/aarch64_cortex-a53/

Build: Requesting version r26809-7c9644a7b5 (kernel 6.6.35)
Downloaded https://sysupgrade.openwrt.org/api/v1/build to /tmp/owut-build-response.json

Requesting build ----------------------
Hash:   7186d20d7fdd4794db7fb7e4ddbd3e85
Status: 202
Detail: queued - 0 ahead of you

Progress   1 (  10s) -----------------------------
Downloaded https://sysupgrade.openwrt.org/api/v1/build/7186d20d7fdd4794db7fb7e4ddbd3e85 to /tmp/owut-build-status.json
Status: 202
Detail: started - setup

Progress   2 (  20s) -----------------------------
Downloaded https://sysupgrade.openwrt.org/api/v1/build/7186d20d7fdd4794db7fb7e4ddbd3e85 to /tmp/owut-build-status.json
Status: 202
Detail: started - calculate_packages_hash

Progress   3 (  30s) -----------------------------
Downloaded https://sysupgrade.openwrt.org/api/v1/build/7186d20d7fdd4794db7fb7e4ddbd3e85 to /tmp/owut-build-status.json
Status: 202
Detail: started - calculate_packages_hash

Progress   4 (  41s) -----------------------------
Downloaded https://sysupgrade.openwrt.org/api/v1/build/7186d20d7fdd4794db7fb7e4ddbd3e85 to /tmp/owut-build-status.json
Status: 202
Detail: started - calculate_packages_hash

Progress   5 (  51s) -----------------------------
Downloaded https://sysupgrade.openwrt.org/api/v1/build/7186d20d7fdd4794db7fb7e4ddbd3e85 to /tmp/owut-build-status.json
Status: 202
Detail: started - building_image

Progress   6 (  61s) -----------------------------
Downloaded https://sysupgrade.openwrt.org/api/v1/build/7186d20d7fdd4794db7fb7e4ddbd3e85 to /tmp/owut-build-status.json
Status: 202
Detail: started - building_image

Progress   7 (  71s) -----------------------------
Downloaded https://sysupgrade.openwrt.org/api/v1/build/7186d20d7fdd4794db7fb7e4ddbd3e85 to /tmp/owut-build-status.json
Status: 202
Detail: started - building_image

Progress   8 (  81s) -----------------------------
Downloaded https://sysupgrade.openwrt.org/api/v1/build/7186d20d7fdd4794db7fb7e4ddbd3e85 to /tmp/owut-build-status.json
Status: 202
Detail: started - building_image

Progress   9 (  91s) -----------------------------
Downloaded https://sysupgrade.openwrt.org/api/v1/build/7186d20d7fdd4794db7fb7e4ddbd3e85 to /tmp/owut-build-status.json
Status: 202
Detail: started - building_image

Progress  10 ( 102s) -----------------------------
Downloaded https://sysupgrade.openwrt.org/api/v1/build/7186d20d7fdd4794db7fb7e4ddbd3e85 to /tmp/owut-build-status.json
Status: 202
Detail: started - building_image

Progress  11 ( 112s) -----------------------------
Downloaded https://sysupgrade.openwrt.org/api/v1/build/7186d20d7fdd4794db7fb7e4ddbd3e85 to /tmp/owut-build-status.json
Status: 200
Detail: done

Build completed in 112 seconds.
  build_at       = 2024-06-27T15:59:31.000000Z
  version_number = SNAPSHOT
  version_code   = r26809-7c9644a7b5
  rootfs_size_mb = default
  init-script    = no-init-script

Image source: https://sysupgrade.openwrt.org/store/7186d20d7fdd4794db7fb7e4ddbd3e85/openwrt-39f1bffe6ee1-mediatek-filogic-bananapi_bpi-r4-squashfs-sysupgrade.itb
Downloaded https://sysupgrade.openwrt.org/store/7186d20d7fdd4794db7fb7e4ddbd3e85/openwrt-39f1bffe6ee1-mediatek-filogic-bananapi_bpi-r4-squashfs-sysupgrade.itb to /tmp/firmware.bin
Image saved : /tmp/firmware.bin
Manifest    : /tmp/firmware-manifest.json
Verifying   : /tmp/firmware.bin (25953063 bytes) against /tmp/firmware.sha256sums
  Saved sha256 matches
  (null)
Checks complete, image is valid.

aaaand

root@BananaPi4:~# sysupgrade --test /tmp/firmware.bin ; echo $?
0
efahl commented 1 week ago

Wow, that's interesting. It appears that sysupgrade is severely broken on your device.

When I download your bin, and check it on my x86 box, it at least tells me that it's the wrong format.

$ sysupgrade --test openwrt-39f1bffe6ee1-mediatek-filogic-bananapi_bpi-r4-squashfs-sysupgrade.itb
Fri Jun 28 08:36:25 PDT 2024 upgrade: Device cwwk-v5 not supported by this image
Fri Jun 28 08:36:25 PDT 2024 upgrade: Supported devices: bananapi,bpi-r4
Fri Jun 28 08:36:25 PDT 2024 upgrade: Invalid image type
Image check failed.

So, let's try grabbing a fresh copy of sysupgrade and redo the test using that:

$ cd /tmp
$ wget https://raw.githubusercontent.com/openwrt/openwrt/main/package/base-files/files/sbin/sysupgrade
$ chmod +x /tmp/sysupgrade

$ /tmp/sysupgrade --test /tmp/firmware.bin
nextgen-networks commented 6 days ago

Hello @efahl, sorry for the late reply.

Here is the output of the commands suggested by you:

...
Build completed in 142 seconds.
  build_at       = 2024-07-01T20:04:41.000000Z
  version_number = SNAPSHOT
  version_code   = r26829-8ef4d78582
  rootfs_size_mb = default
  init-script    = no-init-script

Image source: https://sysupgrade.openwrt.org/store/f5b37d0753acbbe0baf266081c22a01b/openwrt-dfa9437624e4-mediatek-filogic-bananapi_bpi-r4-squashfs-sysupgrade.itb
Downloaded https://sysupgrade.openwrt.org/store/f5b37d0753acbbe0baf266081c22a01b/openwrt-dfa9437624e4-mediatek-filogic-bananapi_bpi-r4-squashfs-sysupgrade.itb to /tmp/firmware.bin
Image saved : /tmp/firmware.bin
Manifest    : /tmp/firmware-manifest.json
Verifying   : /tmp/firmware.bin (27001639 bytes) against /tmp/firmware.sha256sums
  Saved sha256 matches
  (null)
Checks complete, image is valid.
root@BananaPi4:~# sysupgrade --test /tmp/firmware.bin ; echo $?
0
root@BananaPi4:~# cd /tmp
root@BananaPi4:/tmp# wget https://raw.githubusercontent.com/openwrt/openwrt/main/package/base-files/files/sbin/sysupgrade
Downloading 'https://raw.githubusercontent.com/openwrt/openwrt/main/package/base-files/files/sbin/sysupgrade'
Connecting to 185.199.108.133:443
Writing to 'sysupgrade'
sysupgrade           100% |*******************************| 11031   0:00:00 ETA
Download completed (11031 bytes)
root@BananaPi4:/tmp# chmod +x /tmp/sysupgrade
root@BananaPi4:/tmp# /tmp/sysupgrade --test /tmp/firmware.bin
root@BananaPi4:/tmp# /tmp/sysupgrade --test /tmp/firmware.bin ; echo $?
0

Edit: I have a assumtion what causes the "no boot issue" and try to check this tonight (GMT+2). I may be that it is caused by one of the config files/services that are installed. Why? Because also the "regular" sysupgrade will work only if the "keep settings" checkbox gets unchecked. Right after the reboot to default values the config could be restored and system works as intended. This behavior was observed yesterday and may have been caused by tinkering with some installed packages from unofficial sources and/or turning package validation off.

efahl commented 5 days ago

Oh, thanks for that last part, it sort of confirms what I've been thinking.

I've been digging through sysupgrade and all its support scripts trying to figure out what's going on, seems like it could only be the backup step. Since you've got that copy of the sysupgrade script in tmp, let's hack it up and see if we can find the cause.

Try this first: Just run /tmp/sysupgrade -l and see if you get any output.

  1. If yes there is output, then failure must be after the list of files is built, during the backup process. Skip down
  2. If the answer is "no output", that would pinpoint the construction of the backup list as the culprit.
    • Check for files in the list with a space in the name. (The code itself has a comment about this, but no handlers for it, see the sysupgrade script at line 150 in the list_changed_conffiles function.)
    • If there is a file with space in the name, see if you can rename or delete the file, or somehow remove it from the backup list.

If sysupgrade does produce -l/--list-backup output, then it must be dying in the backup creation itself:

  1. Edit /tmp/sysupgrade and add set -x (enables script tracing) at line 2, just after the #!/bin/sh (this is going to generate a ton of output).
  2. Run /tmp/sysupgrade -b /tmp/backup.tgz to see how far it gets. The last 10-20 lines of the trace should be helpful, but maybe it happens before then, you might have to dig through it.
nextgen-networks commented 4 days ago

Short notice: Due to recent problems with missing packages for BPI-R4 (collectd*) and other things thar require my focus within the next days tests needs to be postponed until next week. I'll update that post as soon as new tests have been done.