openwrt / asu

An image on demand server for OpenWrt based distributions
https://sysupgrade.openwrt.org
GNU General Public License v2.0
325 stars 82 forks source link

ASU update issue #344

Closed Tom-Neverwinter closed 2 years ago

Tom-Neverwinter commented 2 years ago

https://github.com/dangowrt/owrt-ubi-installer/commit/13e967448206ea1c56028a90d711027ad45440c8

Unsupported package(s): ip6tables, iptables

Please report the error message and request

{ "board_name": "linksys,e8450-ubi", "target": "mediatek/mt7622", "version": "SNAPSHOT", "packages": [ "auc", "base-files", "blockd", "busybox", "ca-bundle", "dnsmasq", "dropbear", "firewall", "fstools", "ip6tables", "iptables", "kernel", "kmod-fs-exfat", "kmod-fs-ext4", "kmod-fs-f2fs", "kmod-fs-vfat", "kmod-gpio-button-hotplug", "kmod-ipt-offload", "kmod-leds-gpio", "kmod-mt7615-firmware", "kmod-mt7615e", "kmod-mt7915e", "kmod-nls-base", "kmod-nls-cp437", "kmod-nls-iso8859-1", "kmod-nls-utf8", "kmod-usb-storage", "kmod-usb-storage-uas", "kmod-usb3", "libc", "libgcc", "libustream-openssl", "logd", "luci", "luci-app-attendedsysupgrade", "luci-ssl-openssl", "luci-theme-openwrt-2020", "mtd", "netifd", "odhcp6c", "odhcpd-ipv6only", "opkg", "ppp", "ppp-mod-pppoe", "procd", "procd-seccomp", "procd-ujail", "uboot-envtools", "uci", "uclient-fetch", "urandom-seed", "urngd", "wpad-openssl" ], ".anonymous": false, ".name": "request", ".type": "request", ".index": 0 }

jck112 commented 2 years ago

I ran into this problem too with "nftables". It looks like virtual packages (i.e. "Provides: nftables") also need to be added to the redis database in janitor.py, e.g.

def update_target_packages(branch: dict, version: str, target: str):
    ...
    virtual_packages = { virtual_name for p in packages.values() if (virtual_name := p.get("provides")) }
    r.sadd(f"packages-{branch['name']}-{version}-{target}", *(virtual_packages | packages.keys()))
    ...

And something similar in the update_arch_packages function.

jck112 commented 2 years ago

Looks like it may be a bit more complicated since packages can provide multiple virtual packages with specific versions.

Package: erlang
Version: 24.2-1
Depends: libc, libncurses6, librt, zlib, libstdcpp6
Provides: erlang-erts=12.2, erlang-kernel=8.2, erlang-sasl=4.1.1, erlang-stdlib=3.17
Source: feeds/packages/lang/erlang
SourceName: erlang
License: Apache-2.0
LicenseFiles: LICENSE.txt
Section: lang
SourceDateEpoch: 1641764777
CPE-ID: cpe:/a:erlang:erlang
Maintainer: Arnaud Sautaux <arnaud.sautaux@infoteam.ch>
Architecture: aarch64_generic
Installed-Size: 8131004
Filename: erlang_24.2-1_aarch64_generic.ipk
Size: 8094755
SHA256sum: e053884f1dee62773476b70f1e706f44ea685455e91f74cdab9d226b21f72bfe
Description:  Erlang/OTP is a general-purpose programming language and runtime
 environment. Erlang has built-in support for concurrency, distribution
 and fault tolerance.
 .
 This package contains the runtime implementation and a minimal set of
 modules (erts, kernel, sasl & stdlib).
Tom-Neverwinter commented 2 years ago

very interesting. above my head atm though. been spending my time on battletech 3.0 stuff for archival

aparcar commented 2 years ago

@jck112 thanks for looking into this issue. I think a simple split could do the trick, the server shouldn't do any complex dependency tracking but just check if a request is valid before running the ImageBuilder.

What do you think about the following?

virtual_packages = { virtual_name.split("=")[0] for p in packages.values() if (virtual_name := p.get("provides")) }
r.sadd(f"packages-{branch['name']}-{version}-{target}", *(virtual_packages | packages.keys()))
jck112 commented 2 years ago

Makes sense. I think that would only grab the first virtual package; to grab them all you could do something like:

virtual_packages = {
    vpkg.split("=")[0]
    for pkg in packages.values() if (provides := pkg.get("provides"))
    for vpkg in provides.split(", ")
}