search

tonistiigi / xx

Dockerfile cross-compilation helpers
MIT License
338 stars 29 forks source link

fix: add support for deb822 sources.list format #150

Closed panekj closed 3 months ago

panekj commented 4 months ago

Fixes setting arch for new format

crazy-max commented 4 months ago

Is it somewhat related to https://github.com/tonistiigi/xx/pull/132 but to handle new format as well inside source conf?

panekj commented 4 months ago

Is it somewhat related to #132 but to handle new format as well inside source conf?

Yes, currently ubuntu:noble / ubuntu:24.04 fail to limit {archive,security}.ubuntu.com to nativearch https://github.com/lapce/lapce/actions/runs/8196470401/job/22432509922#step:3:2819

I've tested this a bit and it seems that apt doesn't allow mixing formats of .sources and .list, so I had to add repositories in deb822 as well

crazy-max commented 4 months ago

I've tested this a bit and it seems that apt doesn't allow mixing formats of .sources and .list, so I had to add repositories in deb822 as well

Ah right good callout

panekj commented 4 months ago

I've tested this a bit and it seems that apt doesn't allow mixing formats of .sources and .list, so I had to add repositories in deb822 as well

Actually never mind, tested this again in plain shell for ubuntu/debian and it seems to work, but it didn't work for me earlier via xx-apt, will re-test later once I get some sleep.

panekj commented 3 months ago

So it seems when not using new style it fails on ubuntu:noble (only), still not sure why

Details ``` => ERROR [test-cargo test-cargo 2/2] RUN --mount=type=cache,target=/pkg-cache,sharing=locked ./test-cargo.bats 42.2s ------ > [test-cargo test-cargo 2/2] RUN --mount=type=cache,target=/pkg-cache,sharing=locked ./test-cargo.bats: 3.975 1..17 21.43 not ok 1 nocargo 21.43 # (from function `xxrun' in file test_helper.bash, line 54, 21.43 # from function `add' in file test_helper.bash, line 7, 21.43 # in test file test-cargo.bats, line 21) 21.43 # `add clang' failed with status 100 21.43 # Hit:1 http://security.ubuntu.com/ubuntu noble-security InRelease 21.43 # Get:2 http://archive.ubuntu.com/ubuntu noble InRelease [255 kB] 21.43 # Hit:3 http://archive.ubuntu.com/ubuntu noble-updates InRelease 21.43 # Hit:4 http://archive.ubuntu.com/ubuntu noble-backports InRelease 21.43 # Get:5 http://archive.ubuntu.com/ubuntu noble/universe amd64 Packages [19.5 MB] 21.43 # Get:6 http://archive.ubuntu.com/ubuntu noble/main amd64 Packages [1855 kB] 21.43 # Get:7 http://archive.ubuntu.com/ubuntu noble/multiverse amd64 Packages [295 kB] 21.43 # Fetched 21.9 MB in 2s (8938 kB/s) 21.43 # Reading package lists... 21.43 # Reading package lists... 21.43 # Building dependency tree... 21.43 # Reading state information... 21.44 # Package 'libc6-dev' is not installed, so not removed 21.44 # Package 'libgcc-12-dev' is not installed, so not removed 21.44 # 0 upgraded, 0 newly installed, 0 to remove and 7 not upgraded. 21.44 # Hit:1 http://security.ubuntu.com/ubuntu noble-security InRelease 21.44 # Get:2 http://ports.ubuntu.com/ubuntu-ports noble InRelease [255 kB] 21.44 # Hit:3 http://archive.ubuntu.com/ubuntu noble InRelease 21.44 # Hit:4 http://archive.ubuntu.com/ubuntu noble-updates InRelease 21.44 # Get:5 http://ports.ubuntu.com/ubuntu-ports noble-updates InRelease [90.7 kB] 21.44 # Get:6 http://ports.ubuntu.com/ubuntu-ports noble-backports InRelease [90.8 kB] 21.44 # Get:7 http://ports.ubuntu.com/ubuntu-ports noble-security InRelease [90.7 kB] 21.44 # Hit:8 http://archive.ubuntu.com/ubuntu noble-backports InRelease 21.44 # Get:9 http://ports.ubuntu.com/ubuntu-ports noble/main arm64 Packages [1826 kB] 21.44 # Get:10 http://ports.ubuntu.com/ubuntu-ports noble/multiverse arm64 Packages [239 kB] 21.44 # Get:11 http://ports.ubuntu.com/ubuntu-ports noble/universe arm64 Packages [19.3 MB] 21.44 # Get:12 http://ports.ubuntu.com/ubuntu-ports noble/restricted arm64 Packages [97.4 kB] 21.44 # Fetched 22.0 MB in 1s (17.6 MB/s) 21.44 # Reading package lists... 21.44 # Reading package lists... 21.44 # Building dependency tree... 21.44 # Reading state information... 21.44 # Package 'libc6-dev:arm64' is not installed, so not removed 21.44 # Package 'libgcc-12-dev:arm64' is not installed, so not removed 21.44 # 0 upgraded, 0 newly installed, 0 to remove and 7 not upgraded. 21.44 # Hit:1 http://ports.ubuntu.com/ubuntu-ports noble InRelease 21.44 # Hit:2 http://ports.ubuntu.com/ubuntu-ports noble-updates InRelease 21.44 # Hit:3 http://ports.ubuntu.com/ubuntu-ports noble-backports InRelease 21.44 # Hit:4 http://ports.ubuntu.com/ubuntu-ports noble-security InRelease 21.44 # Hit:5 http://security.ubuntu.com/ubuntu noble-security InRelease 21.44 # Hit:6 http://archive.ubuntu.com/ubuntu noble InRelease 21.44 # Get:7 http://ports.ubuntu.com/ubuntu-ports noble/main ppc64el Packages [1780 kB] 21.44 # Hit:8 http://archive.ubuntu.com/ubuntu noble-updates InRelease 21.44 # Hit:9 http://archive.ubuntu.com/ubuntu noble-backports InRelease 21.44 # Get:10 http://ports.ubuntu.com/ubuntu-ports noble/restricted ppc64el Packages [4694 B] 21.44 # Get:11 http://ports.ubuntu.com/ubuntu-ports noble/universe ppc64el Packages [19.1 MB] 21.44 # Get:12 http://ports.ubuntu.com/ubuntu-ports noble/multiverse ppc64el Packages [226 kB] 21.44 # Fetched 21.1 MB in 1s (18.0 MB/s) 21.44 # Reading package lists... 21.44 # Reading package lists... 21.44 # Building dependency tree... 21.44 # Reading state information... 21.44 # Package 'libc6-dev:ppc64el' is not installed, so not removed 21.44 # Package 'libgcc-12-dev:ppc64el' is not installed, so not removed 21.44 # 0 upgraded, 0 newly installed, 0 to remove and 7 not upgraded. 21.44 # Hit:1 http://ports.ubuntu.com/ubuntu-ports noble InRelease 21.44 # Hit:2 http://ports.ubuntu.com/ubuntu-ports noble-updates InRelease 21.44 # Hit:3 http://ports.ubuntu.com/ubuntu-ports noble-backports InRelease 21.44 # Hit:4 http://ports.ubuntu.com/ubuntu-ports noble-security InRelease 21.44 # Hit:5 http://archive.ubuntu.com/ubuntu noble InRelease 21.44 # Hit:6 http://security.ubuntu.com/ubuntu noble-security InRelease 21.44 # Get:7 http://ports.ubuntu.com/ubuntu-ports noble/main s390x Packages [1763 kB] 21.44 # Hit:8 http://archive.ubuntu.com/ubuntu noble-updates InRelease 21.44 # Hit:9 http://archive.ubuntu.com/ubuntu noble-backports InRelease 21.44 # Get:10 http://ports.ubuntu.com/ubuntu-ports noble/restricted s390x Packages [4694 B] 21.44 # Get:11 http://ports.ubuntu.com/ubuntu-ports noble/universe s390x Packages [18.5 MB] 21.44 # Get:12 http://ports.ubuntu.com/ubuntu-ports noble/multiverse s390x Packages [203 kB] 21.44 # Fetched 20.4 MB in 1s (18.4 MB/s) 21.44 # Reading package lists... 21.44 # Reading package lists... 21.44 # Building dependency tree... 21.44 # Reading state information... 21.44 # Package 'libc6-dev:s390x' is not installed, so not removed 21.44 # Package 'libgcc-12-dev:s390x' is not installed, so not removed 21.44 # 0 upgraded, 0 newly installed, 0 to remove and 7 not upgraded. 21.44 # 21.44 # WARNING: apt does not have a stable CLI interface. Use with caution in scripts. 21.44 # 21.44 # E: Conflicting values set for option Signed-By regarding source http://archive.ubuntu.com/ubuntu/ noble: /usr/share/keyrings/ubuntu-archive-keyring.gpg != 21.44 # E: Conflicting values set for option Signed-By regarding source http://archive.ubuntu.com/ubuntu/ noble: /usr/share/keyrings/ubuntu-archive-keyring.gpg != 21.44 # E: The list of sources could not be read. 21.53 not ok 2 install-rustup 21.53 # (from function `xxrun' in file test_helper.bash, line 54, 21.53 # from function `add' in file test_helper.bash, line 7, 21.53 # in test file test-cargo.bats, line 48) 21.53 # `add clang lld curl ca-certificates' failed with status 100 21.53 # 21.53 # WARNING: apt does not have a stable CLI interface. Use with caution in scripts. 21.53 # 21.53 # E: Conflicting values set for option Signed-By regarding source http://archive.ubuntu.com/ubuntu/ noble: /usr/share/keyrings/ubuntu-archive-keyring.gpg != 21.53 # E: Conflicting values set for option Signed-By regarding source http://archive.ubuntu.com/ubuntu/ noble: /usr/share/keyrings/ubuntu-archive-keyring.gpg != 21.53 # E: The list of sources could not be read. 21.66 not ok 3 amd64-hellocargo-rustup 21.66 # (from function `assert_success' in file bats-assert/src/assert.bash, line 114, 21.67 # from function `testHelloCargo' in file test-cargo.bats, line 30, 21.67 # from function `testHelloCargoRustup' in file test-cargo.bats, line 44, 21.67 # in test file test-cargo.bats, line 65) 21.67 # `testHelloCargoRustup' failed 21.67 # 21.67 # -- command failed -- 21.67 # status : 100 21.67 # output (3 lines): 21.67 # E: Conflicting values set for option Signed-By regarding source http://archive.ubuntu.com/ubuntu/ noble: /usr/share/keyrings/ubuntu-archive-keyring.gpg != 21.67 # E: Conflicting values set for option Signed-By regarding source http://archive.ubuntu.com/ubuntu/ noble: /usr/share/keyrings/ubuntu-archive-keyring.gpg != 21.67 # E: The list of sources could not be read. 21.67 # -- 21.67 # 22.65 not ok 4 arm64-hellocargo-rustup 22.65 # (from function `assert_success' in file bats-assert/src/assert.bash, line 114, 22.65 # from function `testHelloCargo' in file test-cargo.bats, line 30, 22.65 # from function `testHelloCargoRustup' in file test-cargo.bats, line 44, 22.65 # in test file test-cargo.bats, line 70) 22.65 # `testHelloCargoRustup' failed 22.65 # 22.65 # -- command failed -- 22.65 # status : 100 22.65 # output (7 lines): 22.65 # + apt install -y --no-install-recommends libc6-dev:arm64 libgcc-8-dev:arm64 22.65 # 22.65 # WARNING: apt does not have a stable CLI interface. Use with caution in scripts. 22.65 # 22.65 # E: Conflicting values set for option Signed-By regarding source http://archive.ubuntu.com/ubuntu/ noble: /usr/share/keyrings/ubuntu-archive-keyring.gpg != 22.65 # E: Conflicting values set for option Signed-By regarding source http://archive.ubuntu.com/ubuntu/ noble: /usr/share/keyrings/ubuntu-archive-keyring.gpg != 22.65 # E: The list of sources could not be read. 22.65 # -- 22.65 # 23.68 not ok 5 arm-hellocargo-rustup 23.68 # (from function `assert_success' in file bats-assert/src/assert.bash, line 114, 23.68 # from function `testHelloCargo' in file test-cargo.bats, line 30, 23.68 # from function `testHelloCargoRustup' in file test-cargo.bats, line 44, 23.68 # in test file test-cargo.bats, line 75) 23.68 # `testHelloCargoRustup' failed 23.68 # 23.68 # -- command failed -- 23.68 # status : 100 23.68 # output (7 lines): 23.68 # + apt install -y --no-install-recommends libc6-dev:armhf libgcc-8-dev:armhf 23.68 # 23.68 # WARNING: apt does not have a stable CLI interface. Use with caution in scripts. 23.68 # 23.68 # E: Conflicting values set for option Signed-By regarding source http://archive.ubuntu.com/ubuntu/ noble: /usr/share/keyrings/ubuntu-archive-keyring.gpg != 23.68 # E: Conflicting values set for option Signed-By regarding source http://archive.ubuntu.com/ubuntu/ noble: /usr/share/keyrings/ubuntu-archive-keyring.gpg != 23.68 # E: The list of sources could not be read. 23.68 # -- 23.68 # 24.68 not ok 6 ppc64le-hellocargo-rustup 24.68 # (from function `assert_success' in file bats-assert/src/assert.bash, line 114, 24.68 # from function `testHelloCargo' in file test-cargo.bats, line 30, 24.68 # from function `testHelloCargoRustup' in file test-cargo.bats, line 44, 24.68 # in test file test-cargo.bats, line 83) 24.68 # `testHelloCargoRustup' failed 24.68 # 24.68 # -- command failed -- 24.68 # status : 100 24.68 # output (7 lines): 24.68 # + apt install -y --no-install-recommends libc6-dev:ppc64el libgcc-8-dev:ppc64el 24.68 # 24.68 # WARNING: apt does not have a stable CLI interface. Use with caution in scripts. 24.68 # 24.68 # E: Conflicting values set for option Signed-By regarding source http://archive.ubuntu.com/ubuntu/ noble: /usr/share/keyrings/ubuntu-archive-keyring.gpg != 24.68 # E: Conflicting values set for option Signed-By regarding source http://archive.ubuntu.com/ubuntu/ noble: /usr/share/keyrings/ubuntu-archive-keyring.gpg != 24.68 # E: The list of sources could not be read. 24.68 # -- 24.68 # 24.70 ok 7 riscv64-hellocargo-rustup # skip RISC-V not supported 25.71 not ok 8 386-hellocargo-rustup 25.71 # (from function `assert_success' in file bats-assert/src/assert.bash, line 114, 25.71 # from function `testHelloCargo' in file test-cargo.bats, line 30, 25.71 # from function `testHelloCargoRustup' in file test-cargo.bats, line 44, 25.71 # in test file test-cargo.bats, line 96) 25.71 # `testHelloCargoRustup' failed 25.71 # 25.71 # -- command failed -- 25.71 # status : 100 25.71 # output (7 lines): 25.72 # + apt install -y --no-install-recommends libc6-dev:i386 libgcc-8-dev:i386 25.72 # 25.72 # WARNING: apt does not have a stable CLI interface. Use with caution in scripts. 25.72 # 25.72 # E: Conflicting values set for option Signed-By regarding source http://archive.ubuntu.com/ubuntu/ noble: /usr/share/keyrings/ubuntu-archive-keyring.gpg != 25.72 # E: Conflicting values set for option Signed-By regarding source http://archive.ubuntu.com/ubuntu/ noble: /usr/share/keyrings/ubuntu-archive-keyring.gpg != 25.72 # E: The list of sources could not be read. 25.72 # -- 25.72 # 25.73 not ok 9 uninstall-rustup 25.74 # (in test file test-cargo.bats, line 101) 25.74 # `rustup self uninstall -y' failed with status 127 25.74 # /work/test-cargo.bats: line 101: rustup: command not found 31.91 not ok 10 install-rustpkg 31.91 # (from function `xxrun' in file test_helper.bash, line 54, 31.91 # from function `add' in file test_helper.bash, line 7, 31.91 # in test file test-cargo.bats, line 106) 31.92 # `add clang lld' failed with status 100 31.92 # 31.92 # WARNING: apt does not have a stable CLI interface. Use with caution in scripts. 31.92 # 31.92 # E: Conflicting values set for option Signed-By regarding source http://archive.ubuntu.com/ubuntu/ noble: /usr/share/keyrings/ubuntu-archive-keyring.gpg != 31.92 # E: Conflicting values set for option Signed-By regarding source http://archive.ubuntu.com/ubuntu/ noble: /usr/share/keyrings/ubuntu-archive-keyring.gpg != 31.92 # E: The list of sources could not be read. 32.05 not ok 11 amd64-hellocargo-rustpkg 32.05 # (from function `assert_success' in file bats-assert/src/assert.bash, line 114, 32.05 # from function `testHelloCargo' in file test-cargo.bats, line 30, 32.05 # in test file test-cargo.bats, line 119) 32.05 # `testHelloCargo' failed 32.05 # 32.05 # -- command failed -- 32.05 # status : 100 32.05 # output (3 lines): 32.05 # E: Conflicting values set for option Signed-By regarding source http://archive.ubuntu.com/ubuntu/ noble: /usr/share/keyrings/ubuntu-archive-keyring.gpg != 32.05 # E: Conflicting values set for option Signed-By regarding source http://archive.ubuntu.com/ubuntu/ noble: /usr/share/keyrings/ubuntu-archive-keyring.gpg != 32.05 # E: The list of sources could not be read. 32.05 # -- 32.05 # 33.03 not ok 12 arm64-hellocargo-rustpkg 33.03 # (from function `assert_success' in file bats-assert/src/assert.bash, line 114, 33.03 # from function `testHelloCargo' in file test-cargo.bats, line 30, 33.03 # in test file test-cargo.bats, line 124) 33.03 # `testHelloCargo' failed 33.03 # 33.03 # -- command failed -- 33.03 # status : 100 33.03 # output (7 lines): 33.03 # + apt install -y --no-install-recommends libc6-dev:arm64 libgcc-8-dev:arm64 33.03 # 33.03 # WARNING: apt does not have a stable CLI interface. Use with caution in scripts. 33.03 # 33.03 # E: Conflicting values set for option Signed-By regarding source http://archive.ubuntu.com/ubuntu/ noble: /usr/share/keyrings/ubuntu-archive-keyring.gpg != 33.03 # E: Conflicting values set for option Signed-By regarding source http://archive.ubuntu.com/ubuntu/ noble: /usr/share/keyrings/ubuntu-archive-keyring.gpg != 33.03 # E: The list of sources could not be read. 33.03 # -- 33.03 # 34.02 not ok 13 arm-hellocargo-rustpkg 34.02 # (from function `assert_success' in file bats-assert/src/assert.bash, line 114, 34.02 # from function `testHelloCargo' in file test-cargo.bats, line 30, 34.02 # in test file test-cargo.bats, line 129) 34.02 # `testHelloCargo' failed 34.02 # 34.02 # -- command failed -- 34.02 # status : 100 34.02 # output (7 lines): 34.02 # + apt install -y --no-install-recommends libc6-dev:armhf libgcc-8-dev:armhf 34.02 # 34.02 # WARNING: apt does not have a stable CLI interface. Use with caution in scripts. 34.02 # 34.02 # E: Conflicting values set for option Signed-By regarding source http://archive.ubuntu.com/ubuntu/ noble: /usr/share/keyrings/ubuntu-archive-keyring.gpg != 34.02 # E: Conflicting values set for option Signed-By regarding source http://archive.ubuntu.com/ubuntu/ noble: /usr/share/keyrings/ubuntu-archive-keyring.gpg != 34.02 # E: The list of sources could not be read. 34.02 # -- 34.02 # 35.02 not ok 14 ppc64le-hellocargo-rustpkg 35.02 # (from function `assert_success' in file bats-assert/src/assert.bash, line 114, 35.02 # from function `testHelloCargo' in file test-cargo.bats, line 30, 35.02 # in test file test-cargo.bats, line 134) 35.02 # `testHelloCargo' failed 35.02 # 35.02 # -- command failed -- 35.02 # status : 100 35.02 # output (7 lines): 35.02 # + apt install -y --no-install-recommends libc6-dev:ppc64el libgcc-8-dev:ppc64el 35.02 # 35.02 # WARNING: apt does not have a stable CLI interface. Use with caution in scripts. 35.02 # 35.02 # E: Conflicting values set for option Signed-By regarding source http://archive.ubuntu.com/ubuntu/ noble: /usr/share/keyrings/ubuntu-archive-keyring.gpg != 35.02 # E: Conflicting values set for option Signed-By regarding source http://archive.ubuntu.com/ubuntu/ noble: /usr/share/keyrings/ubuntu-archive-keyring.gpg != 35.02 # E: The list of sources could not be read. 35.02 # -- 35.02 # 35.04 ok 15 riscv64-hellocargo-rustpkg # skip RISC-V not supported 36.06 not ok 16 386-hellocargo-rustpkg 36.06 # (from function `assert_success' in file bats-assert/src/assert.bash, line 114, 36.06 # from function `testHelloCargo' in file test-cargo.bats, line 30, 36.06 # in test file test-cargo.bats, line 148) 36.06 # `testHelloCargo' failed 36.06 # 36.06 # -- command failed -- 36.06 # status : 100 36.06 # output (7 lines): 36.06 # + apt install -y --no-install-recommends libc6-dev:i386 libgcc-8-dev:i386 36.06 # 36.06 # WARNING: apt does not have a stable CLI interface. Use with caution in scripts. 36.06 # 36.06 # E: Conflicting values set for option Signed-By regarding source http://archive.ubuntu.com/ubuntu/ noble: /usr/share/keyrings/ubuntu-archive-keyring.gpg != 36.06 # E: Conflicting values set for option Signed-By regarding source http://archive.ubuntu.com/ubuntu/ noble: /usr/share/keyrings/ubuntu-archive-keyring.gpg != 36.07 # E: The list of sources could not be read. 36.07 # -- 36.07 # 42.17 ok 17 clean-packages ```
panekj commented 3 months ago

PR in current form works fine, and uses new sources format when detected so I would say it's fine to merge