search

llvm / llvm-project

The LLVM Project is a collection of modular and reusable compiler and toolchain technologies.
http://llvm.org
Other
28.26k stars 11.67k forks source link

llvm.sh fails on bookworm #62475

Closed HappyCerberus closed 4 months ago

HappyCerberus commented 1 year ago

The llvm.sh install script for debian fails to install on Debian Bookworm.

The root cause seems to be apt-get update. In the log it doesn't pick up on the just-added repo pointing to LLVM packages. And naturally, the install fails, as it doesn't see those packages.

I was able to work around it for now by calling the script twice with a manual apt-get update in between.

The log:

#18 8.799 + add-apt-repository 'deb http://apt.llvm.org/unstable/  llvm-toolchain-16 main'
#18 17.45 Hit:1 http://deb.debian.org/debian bookworm InRelease
#18 17.45 Hit:2 http://deb.debian.org/debian bookworm-updates InRelease
#18 17.46 Get:3 http://deb.debian.org/debian-security bookworm-security InRelease [48.0 kB]
#18 22.42 Fetched 48.0 kB in 5s (9238 B/s)
#18 22.42 Reading package lists...
#18 27.05 Repository: 'deb http://apt.llvm.org/unstable/ llvm-toolchain-16 main'
#18 27.05 Description:
#18 27.05 Archive for codename: llvm-toolchain-16 components: main
#18 27.05 More info: http://apt.llvm.org/unstable/
#18 27.05 Adding repository.
#18 27.05 Adding deb entry to /etc/apt/sources.list.d/archive_uri-http_apt_llvm_org_unstable_-bookworm.list
#18 27.05 Adding disabled deb-src entry to /etc/apt/sources.list.d/archive_uri-http_apt_llvm_org_unstable_-bookworm.list
#18 27.14 + apt-get update
#18 27.67 Hit:1 http://deb.debian.org/debian bookworm InRelease
#18 27.67 Hit:2 http://deb.debian.org/debian bookworm-updates InRelease
#18 27.67 Hit:3 http://deb.debian.org/debian-security bookworm-security InRelease
#18 31.90 Reading package lists...
#18 36.50 + PKG='clang-16 lldb-16 lld-16 clangd-16'
#18 36.50 + [[ 1 -eq 1 ]]
#18 36.50 + PKG='clang-16 lldb-16 lld-16 clangd-16 clang-tidy-16 clang-format-16 clang-tools-16 llvm-16-dev lld-16 lldb-16 llvm-16-tools libomp-16-dev libc++-16-dev libc++abi-16-dev libclang-common-16-dev libclang-16-dev libclang-cpp16-dev libunwind-16-dev'
#18 36.50 + test 16 -gt 14
#18 36.50 + PKG='clang-16 lldb-16 lld-16 clangd-16 clang-tidy-16 clang-format-16 clang-tools-16 llvm-16-dev lld-16 lldb-16 llvm-16-tools libomp-16-dev libc++-16-dev libc++abi-16-dev libclang-common-16-dev libclang-16-dev libclang-cpp16-dev libunwind-16-dev libclang-rt-16-dev libpolly-16-dev'
#18 36.50 + apt-get install -y clang-16 lldb-16 lld-16 clangd-16 clang-tidy-16 clang-format-16 clang-tools-16 llvm-16-dev lld-16 lldb-16 llvm-16-tools libomp-16-dev libc++-16-dev libc++abi-16-dev libclang-common-16-dev libclang-16-dev libclang-cpp16-dev libunwind-16-dev libclang-rt-16-dev libpolly-16-dev
#18 36.66 Reading package lists...
#18 41.32 Building dependency tree...
#18 42.04 Reading state information...
#18 42.47 E: Unable to locate package clang-16
#18 42.47 E: Unable to locate package lldb-16
#18 42.47 E: Unable to locate package lld-16
#18 42.47 E: Unable to locate package clangd-16
#18 42.47 E: Unable to locate package clang-tidy-16
#18 42.47 E: Unable to locate package clang-format-16
#18 42.47 E: Unable to locate package clang-tools-16
#18 42.47 E: Unable to locate package llvm-16-dev
#18 42.47 E: Unable to locate package lld-16
#18 42.47 E: Unable to locate package lldb-16
#18 42.47 E: Unable to locate package llvm-16-tools
#18 42.47 E: Unable to locate package libomp-16-dev
#18 42.47 E: Unable to locate package libc++-16-dev
#18 42.47 E: Couldn't find any package by regex 'libc++-16-dev'
#18 42.47 E: Unable to locate package libc++abi-16-dev
#18 42.47 E: Couldn't find any package by regex 'libc++abi-16-dev'
#18 42.47 E: Unable to locate package libclang-common-16-dev
#18 42.47 E: Unable to locate package libclang-16-dev
#18 42.47 E: Unable to locate package libclang-cpp16-dev
#18 42.47 E: Unable to locate package libunwind-16-dev
#18 42.47 E: Unable to locate package libclang-rt-16-dev
#18 42.47 E: Unable to locate package libpolly-16-dev

Here is a link to the log: https://github.com/HappyCerberus/cpp-course/actions/runs/4846401308/jobs/8635920592#step:8:18553

The Dockerfile in question: https://github.com/HappyCerberus/cpp-course/blob/main/.docker/Dockerfile

lesomnus commented 1 year ago

Same problem, but only for libpolly-15-dev.

E: Package 'libclang-rt-15-dev' has no installation candidate
E: Unable to locate package libpolly-15-dev

I also able to install it by run the script twice.

sylvestre commented 1 year ago

I think it is now fixed

HappyCerberus commented 1 year ago

@sylvestre

Can you please link the commit that supposedly fixed it? Because I'm still running into the same problem.

sylvestre commented 1 year ago

I thought this one fixed it: https://github.com/opencollab/llvm-jenkins.debian.net/commit/f30dc7786f1fd9de2ff1041f704836a4274e75a1

I tried to reproduce with bash -c "$(wget -O - https://apt.llvm.org/llvm.sh)" and

wget https://apt.llvm.org/llvm.sh
chmod +x llvm.sh
sudo ./llvm.sh 16

and both worked

HappyCerberus commented 1 year ago

How would that fixed it? Doesn't that only apply when the version is not specified? I was always specifying 16.

Are you testing this on a clean system?

The following Dockerfile still consistently fails for me.

FROM gcc:13.1
ARG TARGETARCH

RUN set -ex && \
    apt-get update && \
    apt-get install -y gpg lsb-release wget software-properties-common

RUN set -ex &&\
    wget https://apt.llvm.org/llvm.sh &&\
    chmod +x llvm.sh &&\
    ./llvm.sh 16 all
sylvestre commented 1 year ago

i just tried your dockerfile but it fails with: E: The repository 'http://deb.debian.org/debian bookworm-updates InRelease' is not signed.

HappyCerberus commented 1 year ago

This is what I get on my machine (and also on Github actions):

#0 0.448 + wget https://apt.llvm.org/llvm.sh
#0 0.449 --2023-06-04 17:16:50--  https://apt.llvm.org/llvm.sh
#0 0.456 Resolving apt.llvm.org (apt.llvm.org)... 199.232.198.49, 199.232.194.49, 2a04:4e42:4d::561, ...
#0 0.458 Connecting to apt.llvm.org (apt.llvm.org)|199.232.198.49|:443... connected.
#0 0.527 HTTP request sent, awaiting response... 200 OK
#0 0.549 Length: 5500 (5.4K) [application/octet-stream]
#0 0.549 Saving to: 'llvm.sh'
#0 0.549 
#0 0.549      0K .....                                                 100% 31.7M=0s
#0 0.549 
#0 0.549 2023-06-04 17:16:50 (31.7 MB/s) - 'llvm.sh' saved [5500/5500]
#0 0.549 
#0 0.550 + chmod +x llvm.sh
#0 0.550 + ./llvm.sh 16 all
#0 0.551 + CURRENT_LLVM_STABLE=15
#0 0.551 + BASE_URL=http://apt.llvm.org
#0 0.551 + needed_binaries=(lsb_release wget add-apt-repository gpg)
#0 0.551 + missing_binaries=()
#0 0.551 + for binary in "${needed_binaries[@]}"
#0 0.551 + which lsb_release
#0 0.552 + for binary in "${needed_binaries[@]}"
#0 0.552 + which wget
#0 0.553 + for binary in "${needed_binaries[@]}"
#0 0.553 + which add-apt-repository
#0 0.553 + for binary in "${needed_binaries[@]}"
#0 0.553 + which gpg
#0 0.554 + [[ 0 -gt 0 ]]
#0 0.554 + LLVM_VERSION=15
#0 0.554 + ALL=0
#0 0.554 ++ lsb_release -is
#0 0.559 + DISTRO=Debian
#0 0.559 ++ lsb_release -sr
#0 0.563 + VERSION=12
#0 0.563 + UBUNTU_CODENAME=
#0 0.563 + CODENAME_FROM_ARGUMENTS=
#0 0.563 + source /etc/os-release
#0 0.563 ++ PRETTY_NAME='Debian GNU/Linux 12 (bookworm)'
#0 0.563 ++ NAME='Debian GNU/Linux'
#0 0.563 ++ VERSION_ID=12
#0 0.563 ++ VERSION='12 (bookworm)'
#0 0.563 ++ VERSION_CODENAME=bookworm
#0 0.563 ++ ID=debian
#0 0.563 ++ HOME_URL=https://www.debian.org/
#0 0.563 ++ SUPPORT_URL=https://www.debian.org/support
#0 0.563 ++ BUG_REPORT_URL=https://bugs.debian.org/
#0 0.563 + DISTRO=debian
#0 0.563 + case ${DISTRO} in
#0 0.563 + [[ 12 (bookworm) == \u\n\s\t\a\b\l\e ]]
#0 0.563 + [[ 12 (bookworm) == \t\e\s\t\i\n\g ]]
#0 0.563 + [[ bookworm == \b\o\o\k\w\o\r\m ]]
#0 0.563 + CODENAME=unstable
#0 0.563 + LINKNAME=
#0 0.563 + '[' 2 -ge 1 ']'
#0 0.563 + '[' 1 '!=' - ']'
#0 0.563 + '[' 16 '!=' all ']'
#0 0.563 + LLVM_VERSION=16
#0 0.563 + OPTIND=2
#0 0.563 + '[' 2 -ge 2 ']'
#0 0.563 + '[' all == all ']'
#0 0.563 + ALL=1
#0 0.563 + OPTIND=3
#0 0.564 + getopts :hm:n: arg
#0 0.564 + [[ 0 -ne 0 ]]
#0 0.564 + declare -A LLVM_VERSION_PATTERNS
#0 0.564 + LLVM_VERSION_PATTERNS[9]=-9
#0 0.564 + LLVM_VERSION_PATTERNS[10]=-10
#0 0.564 + LLVM_VERSION_PATTERNS[11]=-11
#0 0.564 + LLVM_VERSION_PATTERNS[12]=-12
#0 0.564 + LLVM_VERSION_PATTERNS[13]=-13
#0 0.564 + LLVM_VERSION_PATTERNS[14]=-14
#0 0.564 + LLVM_VERSION_PATTERNS[15]=-15
#0 0.564 + LLVM_VERSION_PATTERNS[16]=-16
#0 0.564 + LLVM_VERSION_PATTERNS[17]=
#0 0.564 + '[' '!' _ ']'
#0 0.564 + LLVM_VERSION_STRING=-16
#0 0.564 + [[ -n unstable ]]
#0 0.564 + REPO_NAME='deb http://apt.llvm.org/unstable/  llvm-toolchain-16 main'
#0 0.564 + wget -q --method=HEAD http://apt.llvm.org/unstable
#0 0.753 + [[ ! -f /etc/apt/trusted.gpg.d/apt.llvm.org.asc ]]
#0 0.753 + wget -qO- https://apt.llvm.org/llvm-snapshot.gpg.key
#0 0.753 + tee /etc/apt/trusted.gpg.d/apt.llvm.org.asc
#0 0.859 -----BEGIN PGP PUBLIC KEY BLOCK-----
#0 0.859 Version: GnuPG v1.4.12 (GNU/Linux)
#0 0.859
#0 0.859 mQINBFE9lCwBEADi0WUAApM/mgHJRU8lVkkw0CHsZNpqaQDNaHefD6Rw3S4LxNmM
#0 0.859 EZaOTkhP200XZM8lVdbfUW9xSjA3oPldc1HG26NjbqqCmWpdo2fb+r7VmU2dq3NM
#0 0.859 R18ZlKixiLDE6OUfaXWKamZsXb6ITTYmgTO6orQWYrnW6ckYHSeaAkW0wkDAryl2
#0 0.859 B5v8aoFnQ1rFiVEMo4NGzw4UX+MelF7rxaaregmKVTPiqCOSPJ1McC1dHFN533FY
#0 0.859 Wh/RVLKWo6npu+owtwYFQW+zyQhKzSIMvNujFRzhIxzxR9Gn87MoLAyfgKEzrbbT
#0 0.859 DhqqNXTxS4UMUKCQaO93TzetX/EBrRpJj+vP640yio80h4Dr5pAd7+LnKwgpTDk1
#0 0.859 G88bBXJAcPZnTSKu9I2c6KY4iRNbvRz4i+ZdwwZtdW4nSdl2792L7Sl7Nc44uLL/
#0 0.859 ZqkKDXEBF6lsX5XpABwyK89S/SbHOytXv9o4puv+65Ac5/UShspQTMSKGZgvDauU
#0 0.859 cs8kE1U9dPOqVNCYq9Nfwinkf6RxV1k1+gwtclxQuY7UpKXP0hNAXjAiA5KS5Crq
#0 0.859 7aaJg9q2F4bub0mNU6n7UI6vXguF2n4SEtzPRk6RP+4TiT3bZUsmr+1ktogyOJCc
#0 0.859 Ha8G5VdL+NBIYQthOcieYCBnTeIH7D3Sp6FYQTYtVbKFzmMK+36ERreL/wARAQAB
#0 0.859 tD1TeWx2ZXN0cmUgTGVkcnUgLSBEZWJpYW4gTExWTSBwYWNrYWdlcyA8c3lsdmVz
#0 0.859 dHJlQGRlYmlhbi5vcmc+iQI4BBMBAgAiBQJRPZQsAhsDBgsJCAcDAgYVCAIJCgsE
#0 0.859 FgIDAQIeAQIXgAAKCRAVz00Yr090Ibx+EADArS/hvkDF8juWMXxh17CgR0WZlHCC
#0 0.859 9CTBWkg5a0bNN/3bb97cPQt/vIKWjQtkQpav6/5JTVCSx2riL4FHYhH0iuo4iAPR
#0 0.859 udC7Cvg8g7bSPrKO6tenQZNvQm+tUmBHgFiMBJi92AjZ/Qn1Shg7p9ITivFxpLyX
#0 0.859 wpmnF1OKyI2Kof2rm4BFwfSWuf8Fvh7kDMRLHv+MlnK/7j/BNpKdozXxLcwoFBmn
#0 0.859 l0WjpAH3OFF7Pvm1LJdf1DjWKH0Dc3sc6zxtmBR/KHHg6kK4BGQNnFKujcP7TVdv
#0 0.859 gMYv84kun14pnwjZcqOtN3UJtcx22880DOQzinoMs3Q4w4o05oIF+sSgHViFpc3W
#0 0.859 R0v+RllnH05vKZo+LDzc83DQVrdwliV12eHxrMQ8UYg88zCbF/cHHnlzZWAJgftg
#0 0.859 hB08v1BKPgYRUzwJ6VdVqXYcZWEaUJmQAPuAALyZESw94hSo28FAn0/gzEc5uOYx
#0 0.859 K+xG/lFwgAGYNb3uGM5m0P6LVTfdg6vDwwOeTNIExVk3KVFXeSQef2ZMkhwA7wya
#0 0.859 KJptkb62wBHFE+o9TUdtMCY6qONxMMdwioRE5BYNwAsS1PnRD2+jtlI0DzvKHt7B
#0 0.859 MWd8hnoUKhMeZ9TNmo+8CpsAtXZcBho0zPGz/R8NlJhAWpdAZ1CmcPo83EW86Yq7
#0 0.859 BxQUKnNHcwj2ebkCDQRRPZQsARAA4jxYmbTHwmMjqSizlMJYNuGOpIidEdx9zQ5g
#0 0.859 zOr431/VfWq4S+VhMDhs15j9lyml0y4ok215VRFwrAREDg6UPMr7ajLmBQGau0Fc
#0 0.859 bvZJ90l4NjXp5p0NEE/qOb9UEHT7EGkEhaZ1ekkWFTWCgsy7rRXfZLxB6sk7pzLC
#0 0.859 DshyW3zjIakWAnpQ5j5obiDy708pReAuGB94NSyb1HoW/xGsGgvvCw4r0w3xPStw
#0 0.859 F1PhmScE6NTBIfLliea3pl8vhKPlCh54Hk7I8QGjo1ETlRP4Qll1ZxHJ8u25f/ta
#0 0.859 RES2Aw8Hi7j0EVcZ6MT9JWTI83yUcnUlZPZS2HyeWcUj+8nUC8W4N8An+aNps9l/
#0 0.859 21inIl2TbGo3Yn1JQLnA1YCoGwC34g8QZTJhElEQBN0X29ayWW6OdFx8MDvllbBV
#0 0.859 ymmKq2lK1U55mQTfDli7S3vfGz9Gp/oQwZ8bQpOeUkc5hbZszYwP4RX+68xDPfn+
#0 0.859 M9udl+qW9wu+LyePbW6HX90LmkhNkkY2ZzUPRPDHZANU5btaPXc2H7edX4y4maQa
#0 0.859 xenqD0lGh9LGz/mps4HEZtCI5CY8o0uCMF3lT0XfXhuLksr7Pxv57yue8LLTItOJ
#0 0.859 d9Hmzp9G97SRYYeqU+8lyNXtU2PdrLLq7QHkzrsloG78lCpQcalHGACJzrlUWVP/
#0 0.859 fN3Ht3kAEQEAAYkCHwQYAQIACQUCUT2ULAIbDAAKCRAVz00Yr090IbhWEADbr50X
#0 0.859 OEXMIMGRLe+YMjeMX9NG4jxs0jZaWHc/WrGR+CCSUb9r6aPXeLo+45949uEfdSsB
#0 0.859 pbaEdNWxF5Vr1CSjuO5siIlgDjmT655voXo67xVpEN4HhMrxugDJfCa6z97P0+ML
#0 0.859 PdDxim57uNqkam9XIq9hKQaurxMAECDPmlEXI4QT3eu5qw5/knMzDMZj4Vi6hovL
#0 0.859 wvvAeLHO/jsyfIdNmhBGU2RWCEZ9uo/MeerPHtRPfg74g+9PPfP6nyHD2Wes6yGd
#0 0.859 oVQwtPNAQD6Cj7EaA2xdZYLJ7/jW6yiPu98FFWP74FN2dlyEA2uVziLsfBrgpS4l
#0 0.859 tVOlrO2YzkkqUGrybzbLpj6eeHx+Cd7wcjI8CalsqtL6cG8cUEjtWQUHyTbQWAgG
#0 0.859 5VPEgIAVhJ6RTZ26i/G+4J8neKyRs4vz+57UGwY6zI4AB1ZcWGEE3Bf+CDEDgmnP
#0 0.859 LSwbnHefK9IljT9XU98PelSryUO/5UPw7leE0akXKB4DtekToO226px1VnGp3Bov
#0 0.859 1GBGvpHvL2WizEwdk+nfk8LtrLzej+9FtIcq3uIrYnsac47Pf7p0otcFeTJTjSq3
#0 0.859 krCaoG4Hx0zGQG2ZFpHrSrZTVy6lxvIdfi0beMgY6h78p6M9eYZHQHc02DjFkQXN
#0 0.859 bXb5c6gCHESH5PXwPU4jQEE7Ib9J6sbk7ZT2Mw==
#0 0.859 =j+4q
#0 0.859 -----END PGP PUBLIC KEY BLOCK-----
#0 0.860 ++ apt-key list
#0 0.860 ++ grep -i llvm
#0 0.960 + [[ -z /etc/apt/trusted.gpg.d/apt.llvm.org.asc
#0 0.960 uid           [ unknown] Sylvestre Ledru - Debian LLVM packages <sylvestre@debian.org> ]]
#0 0.960 + add-apt-repository 'deb http://apt.llvm.org/unstable/  llvm-toolchain-16 main'
#0 1.270 Hit:1 http://deb.debian.org/debian bookworm InRelease
#0 1.279 Hit:2 http://deb.debian.org/debian bookworm-updates InRelease
#0 1.298 Hit:3 http://deb.debian.org/debian-security bookworm-security InRelease
#0 1.334 Get:4 http://deb.debian.org/debian bookworm/main amd64 DEP-11 Metadata [4493 kB]
#0 3.325 Fetched 4493 kB in 1s (3871 kB/s)
#0 3.325 Reading package lists...
#0 3.675 Repository: 'deb http://apt.llvm.org/unstable/ llvm-toolchain-16 main'
#0 3.675 Description:
#0 3.675 Archive for codename: llvm-toolchain-16 components: main
#0 3.675 More info: http://apt.llvm.org/unstable/
#0 3.675 Adding repository.
#0 3.675 Adding deb entry to /etc/apt/sources.list.d/archive_uri-http_apt_llvm_org_unstable_-bookworm.list
#0 3.675 Adding disabled deb-src entry to /etc/apt/sources.list.d/archive_uri-http_apt_llvm_org_unstable_-bookworm.list
#0 3.687 + apt-get update
#0 3.759 Hit:1 http://deb.debian.org/debian bookworm InRelease
#0 3.776 Hit:2 http://deb.debian.org/debian bookworm-updates InRelease
#0 3.795 Hit:3 http://deb.debian.org/debian-security bookworm-security InRelease
#0 3.883 Reading package lists...
#0 4.248 + PKG='clang-16 lldb-16 lld-16 clangd-16'
#0 4.248 + [[ 1 -eq 1 ]]
#0 4.248 + PKG='clang-16 lldb-16 lld-16 clangd-16 clang-tidy-16 clang-format-16 clang-tools-16 llvm-16-dev lld-16 lldb-16 llvm-16-tools libomp-16-dev libc++-16-dev libc++abi-16-dev libclang-common-16-dev libclang-16-dev libclang-cpp16-dev libunwind-16-dev'
#0 4.248 + test 16 -gt 14
#0 4.248 + PKG='clang-16 lldb-16 lld-16 clangd-16 clang-tidy-16 clang-format-16 clang-tools-16 llvm-16-dev lld-16 lldb-16 llvm-16-tools libomp-16-dev libc++-16-dev libc++abi-16-dev libclang-common-16-dev libclang-16-dev libclang-cpp16-dev libunwind-16-dev libclang-rt-16-dev libpolly-16-dev'
#0 4.248 + apt-get install -y clang-16 lldb-16 lld-16 clangd-16 clang-tidy-16 clang-format-16 clang-tools-16 llvm-16-dev lld-16 lldb-16 llvm-16-tools libomp-16-dev libc++-16-dev libc++abi-16-dev libclang-common-16-dev libclang-16-dev libclang-cpp16-dev libunwind-16-dev libclang-rt-16-dev libpolly-16-dev
#0 4.251 Reading package lists...
#0 4.611 Building dependency tree...
#0 4.701 Reading state information...
#0 4.762 E: Unable to locate package clang-16
#0 4.762 E: Unable to locate package lldb-16
#0 4.762 E: Unable to locate package lld-16
#0 4.762 E: Unable to locate package clangd-16
#0 4.762 E: Unable to locate package clang-tidy-16
#0 4.762 E: Unable to locate package clang-format-16
#0 4.762 E: Unable to locate package clang-tools-16
#0 4.762 E: Unable to locate package llvm-16-dev
#0 4.762 E: Unable to locate package lld-16
#0 4.762 E: Unable to locate package lldb-16
#0 4.762 E: Unable to locate package llvm-16-tools
#0 4.762 E: Unable to locate package libomp-16-dev
#0 4.762 E: Unable to locate package libc++-16-dev
#0 4.762 E: Couldn't find any package by regex 'libc++-16-dev'
#0 4.762 E: Unable to locate package libc++abi-16-dev
#0 4.762 E: Couldn't find any package by regex 'libc++abi-16-dev'
#0 4.762 E: Unable to locate package libclang-common-16-dev
#0 4.762 E: Unable to locate package libclang-16-dev
#0 4.762 E: Unable to locate package libclang-cpp16-dev
#0 4.762 E: Unable to locate package libunwind-16-dev
#0 4.762 E: Unable to locate package libclang-rt-16-dev
#0 4.762 E: Unable to locate package libpolly-16-dev
sylvestre commented 1 year ago

ok, it is probably because bookworm is now a distro and not unstable

sylvestre commented 1 year ago

I think I fixed it. Bookworm is now in a new branch. Could you please check? Thanks

HappyCerberus commented 1 year ago

@sylvestre

OK, I took some time and figured out the actual problem. The sources file is empty:

drwxr-xr-x 1 root root 4096 Jun  6 18:11 .
drwxr-xr-x 1 root root 4096 Jun  6 18:11 ..
-rw-r--r-- 1 root root    0 Jun  6 18:11 archive_uri-http_apt_llvm_org_bookworm_-bookworm.list
-rw-r--r-- 1 root root  443 Apr 11 00:00 debian.sources

The reason is add-apt-repository, on the first run it only creates an empty file and fills it on the second run, without reporting any errors or warnings.

I don't know if this is a debian bug or something weird with the llvm repo (probably both).

lesomnus commented 1 year ago

It still happens on the latest gcc:13 image. I think this issue should be re-opened.

sylvestre commented 1 year ago

I can reproduce. Seems like a problem with add-apt-repository;

As workaround 

FROM gcc:13.1
ARG TARGETARCH

RUN set -ex && \
    apt-get update && \
    apt-get install -y gpg lsb-release wget software-properties-common

RUN set -ex &&\
    echo "deb http://apt.llvm.org/bookworm/ llvm-toolchain-bookworm-16 main" > /etc/apt/sources.list.d/apt.llvm.org.list &&\
    wget -qO- https://apt.llvm.org/llvm-snapshot.gpg.key |  tee /etc/apt/trusted.gpg.d/apt.llvm.org.asc &&\
    apt update &&\
    apt-get install -y clang-16 lldb-16 lld-16 clangd-16 clang-tidy-16 clang-format-16 clang-tools-16 llvm-16-dev lld-16 lldb-16 llvm-16-tools libomp-16-dev libc++-16-dev libc++abi-16-dev libclang-common-16-dev libclang-16-dev libclang-cpp16-dev libunwind-16-dev libclang-rt-16-dev libpolly-16-dev
lesomnus commented 1 year ago

How about using echo "${REPO_NAME}" > /etc/apt/sources.list.d/apt.llvm.org.list instead of using add-apt-repository in llvm.sh? Is there any specific reason why we have to use add-apt-repository?

mpickering commented 4 months ago

Bumping this issue as it is still a problem.

sylvestre commented 4 months ago

I will have a look, sorry, i dropped it

mpickering commented 4 months ago

I have worked around this problem in the same way as https://github.com/vringar/LibAFL/commit/bb6b17eefb0f657fdf6f5196ce6edbeb1395c0c7

It would be a shame if this simpler script stopped working for future debian versions as it's a very convenient way to install the LLVM toolchain in docker images.

sylvestre commented 4 months ago

I added the workaround to run add-apt-repository twice when it is running Debian bookworm: https://github.com/opencollab/llvm-jenkins.debian.net/blob/master/llvm.sh#L166-L170'

This Dockerfile works for me:


FROM gcc:13.1
ARG TARGETARCH

RUN set -ex && \
    apt-get update && \
    apt-get install -y gpg lsb-release wget software-properties-common

RUN set -ex &&\
    wget https://apt.llvm.org/llvm.sh &&\
    chmod +x llvm.sh &&\
    ./llvm.sh && clang-18 --version
mpickering commented 3 months ago

Thanks @sylvestre