google / fuzzbench

FuzzBench - Fuzzer benchmarking as a service.
https://google.github.io/fuzzbench/
Apache License 2.0
1.1k stars 268 forks source link

aflplusplus_optimal dockerfile failure to get clang-12 #1070

Closed inferno-chromium closed 3 years ago

inferno-chromium commented 3 years ago

make test-run-aflplusplus_optimal-freetype2-2017

#6 16.56 gpg:               imported: 1  (RSA: 1)
#6 17.07 Get:1 https://apt.llvm.org/xenial llvm-toolchain-xenial InRelease [4218 B]
#6 17.08 Hit:2 http://archive.ubuntu.com/ubuntu xenial InRelease
#6 17.08 Hit:3 http://security.ubuntu.com/ubuntu xenial-security InRelease
#6 17.08 Get:4 http://ppa.launchpad.net/ubuntu-toolchain-r/test/ubuntu xenial InRelease [17.5 kB]
#6 17.21 Hit:5 http://archive.ubuntu.com/ubuntu xenial-updates InRelease
#6 17.23 Get:6 https://apt.llvm.org/xenial llvm-toolchain-xenial/main amd64 Packages [10.6 kB]
#6 17.25 Get:7 https://apt.llvm.org/xenial llvm-toolchain-xenial/main i386 Packages [1597 B]
#6 17.35 Hit:8 http://archive.ubuntu.com/ubuntu xenial-backports InRelease
#6 17.48 Hit:9 http://ppa.launchpad.net/git-core/ppa/ubuntu xenial InRelease
#6 17.97 Get:10 http://ppa.launchpad.net/ubuntu-toolchain-r/test/ubuntu xenial/main amd64 Packages [107 kB]
#6 18.65 Get:11 http://ppa.launchpad.net/ubuntu-toolchain-r/test/ubuntu xenial/main i386 Packages [105 kB]
#6 19.10 Fetched 246 kB in 2s (107 kB/s)
#6 19.10 Reading package lists...
#6 20.41 Reading package lists...
#6 21.72 Building dependency tree...
#6 21.93 Reading state information...
#6 22.03 E: Unable to locate package clang-12
#6 22.03 E: Unable to locate package clangd-12
#6 22.03 E: Unable to locate package clang-tools-12
#6 22.03 E: Unable to locate package libc++1-12
#6 22.03 E: Couldn't find any package by regex 'libc++1-12'
#6 22.03 E: Unable to locate package libc++-12-dev
#6 22.03 E: Couldn't find any package by regex 'libc++-12-dev'
#6 22.03 E: Unable to locate package libc++abi1-12
#6 22.03 E: Couldn't find any package by regex 'libc++abi1-12'
#6 22.03 E: Unable to locate package libc++abi-12-dev
#6 22.03 E: Couldn't find any package by regex 'libc++abi-12-dev'
#6 22.03 E: Unable to locate package libclang1-12
#6 22.03 E: Unable to locate package libclang-12-dev
#6 22.03 E: Unable to locate package libclang-common-12-dev
#6 22.03 E: Unable to locate package libclang-cpp12
#6 22.03 E: Unable to locate package libclang-cpp12-dev
#6 22.03 E: Unable to locate package liblld-12
#6 22.03 E: Unable to locate package liblld-12-dev
#6 22.03 E: Unable to locate package liblldb-12
#6 22.03 E: Unable to locate package liblldb-12-dev
#6 22.03 E: Unable to locate package libllvm12
#6 22.03 E: Unable to locate package libomp-12-dev
#6 22.03 E: Unable to locate package libomp5-12
#6 22.03 E: Unable to locate package lld-12
#6 22.03 E: Unable to locate package lldb-12
#6 22.03 E: Unable to locate package llvm-12
#6 22.03 E: Unable to locate package llvm-12-dev
#6 22.03 E: Unable to locate package llvm-12-runtime
#6 22.03 E: Unable to locate package llvm-12-tools
------
executor failed running [/bin/sh -c apt-get update &&     apt-get install -y wget libstdc++-5-dev libexpat1-dev &&     apt-get install -y apt-utils apt-transport-https ca-certificates &&     echo deb http://apt.llvm.org/xenial/ llvm-toolchain-xenial main >> /etc/apt/sources.list &&     echo deb http://ppa.launchpad.net/ubuntu-toolchain-r/test/ubuntu xenial main >> /etc/apt/sources.list &&     wget -O - https://apt.llvm.org/llvm-snapshot.gpg.key | apt-key add - &&     apt-key adv --recv-keys --keyserver keyserver.ubuntu.com 1E9377A2BA9EF27F &&     apt-get update &&     apt-get install -y clang-12 clangd-12 clang-tools-12 libc++1-12 libc++-12-dev       libc++abi1-12 libc++abi-12-dev libclang1-12 libclang-12-dev libclang-common-12-dev       libclang-cpp12 libclang-cpp12-dev liblld-12 liblld-12-dev liblldb-12       liblldb-12-dev libllvm12 libomp-12-dev libomp5-12 lld-12 lldb-12       llvm-12 llvm-12-dev llvm-12-runtime llvm-12-tools &&     apt-get install -y gcc-9 g++-9]: exit code: 100
inferno-chromium commented 3 years ago

@vanhauser-thc fyi to fix sometime, i was trying to debug ci issue with aflplusplus_optimal and aflplusplus_eclipser, but ran into this. should be something minor in /etc/apt/sources.list.

inferno-chromium commented 3 years ago

don't see clang-12 in there - https://apt.llvm.org/xenial/dists/

yifengchen-cc commented 3 years ago

https://github.com/google/fuzzbench/blob/master/fuzzers/aflplusplus_optimal/builder.Dockerfile#L22 change llvm-toolchain-xenial to llvm-toolchain-xenial-12 can fix this problem

inferno-chromium commented 3 years ago

https://github.com/google/fuzzbench/blob/master/fuzzers/aflplusplus_optimal/builder.Dockerfile#L22 change llvm-toolchain-xenial to llvm-toolchain-xenial-12 can fix this problem

nope it does not, see my last comment, there is no 12 support in https://apt.llvm.org/xenial/dists/

yifengchen-cc commented 3 years ago

nope it does not, see my last comment, there is no 12 support in https://apt.llvm.org/xenial/dists/

That's weird, I used this method to create a local experiment successfully. It has llvm-toolchain-xenial-12 in https://apt.llvm.org/

Precise, Quantal, Raring, Saucy, Utopic, Artful, Cosmic, Eoan and Trusty are no longer supported by Ubuntu. Repo remains available

Xenial LTS (16.04) - Last update : Wed, 03 Feb 2021 04:56:43 UTC / Revision: 20210203020828+af2cbe8effca
deb http://apt.llvm.org/xenial/ llvm-toolchain-xenial main
deb-src http://apt.llvm.org/xenial/ llvm-toolchain-xenial main
# 11
deb http://apt.llvm.org/xenial/ llvm-toolchain-xenial-11 main
deb-src http://apt.llvm.org/xenial/ llvm-toolchain-xenial-11 main
# 12
deb http://apt.llvm.org/xenial/ llvm-toolchain-xenial-12 main
deb-src http://apt.llvm.org/xenial/ llvm-toolchain-xenial-12 main
vanhauser-thc commented 3 years ago
Feb 01st 2021 - Snapshot becomes 13, branch 12 created

that why it breaks. happened too when snapshot became 12 from 11. there is no -X branch until the snapshot switched to Y... will fix it :)

vanhauser-thc commented 3 years ago

ok ... they switched snapshot to provide 13, but did not create 12. so there is currently no 12 available. what do? I'd propose waiting for a few days and see if they add it, and if not downgrade to 11?

vanhauser-thc commented 3 years ago

Now it is there - will update with the next experiment (Friday)

inferno-chromium commented 3 years ago

this is fixed with @vanhauser-thc cl