Multiple packages: package size mismatches

[Traumflug]

Environment: OpenWrt 19.07.0-rc2 r10775-db8345d8e4 on a TP-Link TL-WDR3600 (target ar71xx generic)

Description: attempting to install some packages results in failure:

opkg install xinetd openssl-util ca-certificates
Installing xinetd (2.3.15-5) to root...
Downloading http://downloads.openwrt.org/releases/19.07.0-rc2/packages/mips_24kc/packages/xinetd_2.3.15-5_mips_24kc.ipk
Installing openssl-util (1.1.1d-2) to root...
Downloading http://downloads.openwrt.org/releases/19.07.0-rc2/packages/mips_24kc/base/openssl-util_1.1.1d-2_mips_24kc.ipk
Installing ca-certificates (20190110-1) to root...
Downloading http://downloads.openwrt.org/releases/19.07.0-rc2/packages/mips_24kc/base/ca-certificates_20190110-1_all.ipk
Collected errors:
 * opkg_install_pkg: Package size mismatch: xinetd is 49992 bytes, expecting 49989 bytes
 * opkg_install_cmd: Cannot install package xinetd.
 * opkg_install_pkg: Package size mismatch: openssl-util is 239246 bytes, expecting 239244 bytes
 * opkg_install_cmd: Cannot install package openssl-util.
 * opkg_install_pkg: Package size mismatch: ca-certificates is 125595 bytes, expecting 125601 bytes
 * opkg_install_cmd: Cannot install package ca-certificates.

[BKPepe]

You should run opkg update before doing opkg install

[Traumflug]

I'm pretty sure I updated before the installation attempt. Nevertheless, doing the same today again worked out. Apparently just a temporary flaw.

Thanks for helping.

[brianjmurrell]

This happens while building images with the imagebuilder also. So I don't think the suggestion to run opkg update is valid in that case, since i would imagine the imagebuilder does that, doesn't it?

[exquo]

When this happens with imagebuilder, can do make clean. I guess make .. doesn't do opkg update before downloading, maybe it should.

[dangowrt]

Looks more like a reproducible-builds issue to me. @lynxis @aparcar ?

[lynxis]

@dangowrt I've no idea what happened here. I would expect those package are reproducible and would guess there is more likely a build pipeline problem.

[brianjmurrell]

When this happens with imagebuilder, can do make clean. I guess make .. doesn't do opkg update before downloading, maybe it should.

This sounds more like a hack-around than a solution. Having to run make clean before make in any Makefile driven build environment is almost always a hack around a broken build system.

The entire point of make in fact is to only do as little rebuilding as is needed given what has changed in the source tree to get a build. make is meant to not have to rebuild the whole world every time. make clean defeats this goal.

[exquo]

This sounds more like a hack-around than a solution.

Yes, I guess the real solution would be to include opkg update as a part of make image.

[aparcar]

This sounds more like a hack-around than a solution.

Yes, I guess the real solution would be to include opkg update as a part of make image.

I remember that this is done already. Is the mismatch possible due to the upstream repo bit being updated in a atomic way?

[sayed-amir]

Hi For image builder, this can be fixed by "make clean". It looks previously downloaded packages are changed online. so, after "clean"ing, image builder download the recent version. This works for me.

[brianjmurrell]

@hoseiniamir make clean defeats the purpose of make which is to only take actions to update things that have changed dependencies.

If one were to need/supposed to make clean for every run, one could just do away with the extra complications that make altogether and just write a start-to-finish shell-script that nukes the environment clean at the start and just always rebuilds everything.

Bur one doesn't do that because make is meant to save the time not doing what doesn't need doing because some dependencies have not changed.

Moreover, having to make clean with the imagebuilder is absolutely a regression. This was not needed prior to 19.07.

[aparcar]

@jow- is it possible that different SDKs rebuild the packages with the same version/releases but cause different checksums due to unreproducibility? If so that's another reason for incremental package builds

[leggewie]

@brianjmurrell I agree with your point but wanted to chirp in and say that I just ran into this issue with openwrt-imagebuilder-18.06.7-ramips-rt305x.Linux-x86_64

Further research suggests that the regression was apparently introduced in 19.07 and then backported to 18.06.5

[zorun]

There is already a bug report about the imagebuilder issue: https://bugs.openwrt.org/index.php?do=details&task_id=2690

The original issue happens on the device and is likely unrelated (you need to run opkg update before installing packages)

[zorun]

To be more clear:

• in this issue, an old package index is used to check the size/checksum of newer packages. Failure is expected.
• in the imagebuilder, a new package index is used to check the size/checksum of old packages that had been downloaded before. This is a bug, opkg should delete the old packages in this case (see ticket).

[aparcar]

I think this is an issue of reproducibility and should be fixed via https://patchwork.ozlabs.org/project/openwrt/list/?series=195728

If checksums don't change opkg should be fine event if the index wasn't updated.

[zorun]

Reproducible packages would indeed not trigger this bug, but it's still a bug (the imagebuilder one). And you can't expect all packages to stay bit-to-bit identical: things like toolchain changes or dependencies updates could change the content of a package while it keeps the same version number.

Btw, since this is about the imagebuilder bug, please use the bug tracker linked above. This ticket can be closed.

[NukedBart]

same issue trying to install tor, opkg update reported an error when some of the package lists cannot be downloaded. error code 404 at https://openwrt.proxy.ustclug.org/snapshots/packages/arm_cortex-a7_neon-vfpv4/helloworld/Packages.gz https://openwrt.proxy.ustclug.org/snapshots/packages/arm_cortex-a7_neon-vfpv4/lienol/Packages.gz