search

google / fuzzbench

FuzzBench - Fuzzer benchmarking as a service.
https://google.github.io/fuzzbench/
Apache License 2.0
1.09k stars 266 forks source link

Error when running a local experiment on Fuzzbench #1484

Open amie0 opened 2 years ago

amie0 commented 2 years ago

hi I cant run it locally make presubmit was successful.Then I wanted to run the experiment for multiple trials and I tried it by placing the experiment-config.yaml file. Then run the command: `PYTHONPATH=. python3 experiment/run_experiment.py \

--experiment-config experiment-config.yaml \ --benchmarks freetype2-2017 bloaty_fuzz_target \ --experiment-name testexperiment \ --fuzzers afl libfuzzer unfortunatelly
INFO:root:Starting experiment. ERROR:root:Executed command: "make -j base-image worker" returned: 2. Extras: {'output': 'arted\n Running setup.py install for importlab: finished with status \'done\'\n Running setup.py install for serverfiles: started\n Running setup.py install for serverfiles: finished with status \'done\'\n Running setup.py install for mozinfo: started\n Running setup.py install for mozinfo: finished with status \'done\'\n Running setup.py install for alembic: started\n Running setup.py install for alembic: finished with status \'done\'\n Running setup.py install for google-cloud-profiler: started\n Running setup.py install for google-cloud-profiler: finished with status \'done\'\nSuccessfully installed AnyQt-0.2.0 Jinja2-2.11.3 Mako-1.2.2 MarkupSafe-2.0.1 Orange3-3.28.0 PyQt5-5.15.7 PyQt5-Qt5-5.15.2 PyQt5-sip-12.11.0 PyQtWebEngine-5.15.6 PyQtWebEngine-Qt5-5.15.2 PyYAML-5.4 SecretStorage-3.3.3 alembic-1.4.0 astroid-2.6.6 attrs-22.1.0 baycomp-1.0.2 bottleneck-1.3.5 cachecontrol-0.12.11 cachetools-4.2.4 certifi-2022.6.15 cffi-1.15.1 chardet-5.0.0 charset-normalizer-2.1.1 click-8.1.3 clusterfuzz-0.0.1a0 commonmark-0.9.1 cryptography-37.0.4 cycler-0.11.0 dictdiffer-0.9.0 distro-1.7.0 docutils-0.19 et-xmlfile-1.1.0 fonttools-4.37.1 google-api-core-1.32.0 google-api-python-client-2.5.0 google-auth-1.30.1 google-auth-httplib2-0.1.0 google-auth-oauthlib-0.5.2 google-cloud-core-1.7.3 google-cloud-datastore-1.15.5 google-cloud-error-reporting-1.1.2 google-cloud-logging-1.15.1 google-cloud-monitoring-2.11.1 google-cloud-ndb-1.11.1 google-cloud-profiler-4.0.0 google-cloud-secret-manager-2.4.0 google-cloud-storage-2.2.1 google-crc32c-1.3.0 google-resumable-media-2.3.3 googleapis-common-protos-1.56.4 grpc-google-iam-v1-0.12.4 grpcio-1.47.0 h11-0.13.0 httpcore-0.12.3 httplib2-0.20.4 httpx-0.16.1 idna-3.3 importlab-0.7 importlib-metadata-4.12.0 iniconfig-1.1.1 isort-5.10.1 jeepney-0.8.0 joblib-1.1.0 keyring-23.8.2 keyrings.alt-4.1.1 kiwisolver-1.4.4 lazy-object-proxy-1.7.1 libcst-0.4.7 lockfile-0.12.2 matplotlib-3.5.3 mccabe-0.6.1 mozfile-2.1.0 mozinfo-1.2.2 mozprocess-1.3.0 msgpack-1.0.4 mypy-extensions-0.4.3 networkx-2.8.6 ninja-1.10.2.3 numpy-1.21.0 oauth2client-4.1.3 oauthlib-3.2.0 openTSNE-0.6.2 openpyxl-3.0.10 orange-canvas-core-0.1.27 orange-widget-base-4.17.0 packaging-21.3 pandas-1.2.4 patsy-0.5.2 pillow-9.2.0 pluggy-0.13.1 proto-plus-1.22.1 protobuf-3.20.1 psutil-5.9.0 psycopg2-binary-2.8.4 py-1.11.0 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pyfakefs-3.7.1 pylint-2.7.4 pymemcache-3.5.2 pyparsing-3.0.9 pyqtgraph-0.12.4 pytest-6.1.2 python-dateutil-2.8.1 python-editor-1.0.4 python-louvain-0.16 pytype-2021.4.15 pytz-2019.3 qasync-0.23.0 redis-3.5.3 requests-2.28.1 requests-oauthlib-1.3.1 rfc3986-1.5.0 rq-1.4.3 rsa-4.9 scikit-learn-1.1.2 scikit-posthocs-0.6.2 scipy-1.6.2 seaborn-0.11.1 serverfiles-0.3.1 six-1.16.0 sniffio-1.2.0 sqlalchemy-1.3.19 statsmodels-0.13.2 threadpoolctl-3.1.0 toml-0.10.2 typed-ast-1.5.4 typing-extensions-4.3.0 typing-inspect-0.8.0 uritemplate-3.0.1 urllib3-1.26.12 wrapt-1.12.1 xlrd-2.0.1 xlsxwriter-3.0.3 yapf-0.30.0 zipp-3.8.1\nsource .venv/bin/activate && PYTHONPATH=. python3 docker/generate_makefile.py docker/generated.mk\ndocker pull ubuntu:xenial\nxenial: Pulling from library/ubuntu\nDigest: sha256:91bd29a464fdabfcf44e29e1f2a5f213c6dfa750b6290e40dd6998ac79da3c41\nStatus: Image is up to date for ubuntu:xenial\ndocker build \\n--tag gcr.io/fuzzbench/base-image \\n--build-arg BUILDKIT_INLINE_CACHE=1 \\n--cache-from gcr.io/fuzzbench/base-image \\n--file docker/base-image/Dockerfile \\n.\n\n#2 [internal] load .dockerignore\n#2 digest: sha256:16d0df9858d81eca73975e6c46a8b6ead0852f543011686baa9afee625d2b71c\n#2 name: "[internal] load .dockerignore"\n#2 started: 2022-09-01 03:09:54.577877083 +0000 UTC\n#2 completed: 2022-09-01 03:09:54.577918759 +0000 UTC\n#2 duration: 41.676µs\n#2 started: 2022-09-01 03:09:54.57821986 +0000 UTC\n#1 ...\n\n#1 [internal] load build definition from Dockerfile\n#1 digest: sha256:57433ba08e23227e9d39aedbe741164987f5e7d74e067286c5ebc39de04a048a\n#1 name: "[internal] load build definition from Dockerfile"\n#1 started: 2022-09-01 03:09:54.536909796 +0000 UTC\n#1 completed: 2022-09-01 03:09:54.536962975 +0000 UTC\n#1 duration: 53.179µs\n#1 started: 2022-09-01 03:09:54.537161007 +0000 UTC\n#1 completed: 2022-09-01 03:09:58.688779892 +0000 UTC\n#1 duration: 4.151618885s\n#1 transferring dockerfile: 2.00kB done\n\n\n#2 [internal] load .dockerignore\n#2 transferring context: 143B done\n#2 completed: 2022-09-01 03:10:00.139782363 +0000 UTC\n#2 duration: 5.561562503s\n\n\n#5 [1/7] FROM docker.io/library/ubuntu:xenial\n#5 digest: sha256:14dd403f25ae9e519bd44f27dc17af7ce80f5a4f9f48ca88108364e67ec5d360\n#5 name: "[1/7] FROM docker.io/library/ubuntu:xenial"\n#5 started: 2022-09-01 03:10:00.658333095 +0000 UTC\n#5 completed: 2022-09-01 03:10:00.659016816 +0000 UTC\n#5 duration: 683.721µs\n\n\n#8 [internal] load build context\n#8 digest: sha256:cfa463da6dbd3e62ce7ad89a34dfe2db0a41eff97ee883e6df3081c8981aa9cf\n#8 name: "[internal] load build context"\n#8 started: 2022-09-01 03:10:00.658403926 +0000 UTC\n#8 completed: 2022-09-01 03:10:00.658551012 +0000 UTC\n#8 duration: 147.086µs\n\n\n#3 [internal] load metadata for docker.io/library/ubuntu:xenial\n#3 digest: sha256:f62bae18ed588840f045c881b2502661f2a89578a2f8f99b83f5200ee83dd25b\n#3 name: "[internal] load metadata for docker.io/library/ubuntu:xenial"\n#3 started: 2022-09-01 03:10:00.655364511 +0000 UTC\n#3 completed: 2022-09-01 03:10:00.655844253 +0000 UTC\n#3 duration: 479.742µs\n\n\n#4 importing cache manifest from gcr.io/fuzzbench/base-image\n#4 digest: sha256:9b220ebb4d4daceed1b96df9b6dca076e762f1cd612574f010b82d5b9e5ad896\n#4 name: "importing cache manifest from gcr.io/fuzzbench/base-image"\n#4 started: 2022-09-01 03:10:00.657580785 +0000 UTC\n#4 completed: 2022-09-01 03:10:03.905354032 +0000 UTC\n#4 duration: 3.247773247s\n\n\n#8 [internal] load build context\n#8 started: 2022-09-01 03:10:03.906843654 +0000 UTC\n#8 transferring context:\n#8 transferring context: 608B done\n#8 completed: 2022-09-01 03:10:07.458310793 +0000 UTC\n#8 duration: 3.551467139s\n\n\n#6 [2/7] RUN apt-get update -y && apt-get install -y build-essential ...\n#6 digest: sha256:3a1b0960f17eed529eaae93dc8fd3554f2743dd584d8fa569cdbe6bc734371ac\n#6 name: "[2/7] RUN apt-get update -y && apt-get install -y build-essential rsync curl zlib1g-dev libncurses5-dev libgdbm-dev libnss3-dev libssl-dev libreadline-dev libffi-dev virtualenv libbz2-dev liblzma-dev libsqlite3-dev"\n#6 started: 2022-09-01 03:10:07.653181139 +0000 UTC\n#6 completed: 2022-09-01 03:10:07.653181139 +0000 UTC\n#6 duration: 0s\n#6 cached: true\n\n\n#7 [3/7] RUN cd /tmp/ && curl -O https://www.python.org/ftp/python/3.8....\n#7 digest: sha256:c5b9d1fb3371e84dbc60b76e7b2f8d5913bc93bcd4910ab65daa03e7b2ec4bb0\n#7 name: "[3/7] RUN cd /tmp/ && curl -O https://www.python.org/ftp/python/3.8.6/Python-3.8.6.tar.xz && tar -xvf Python-3.8.6.tar.xz && cd Python-3.8.6 && ./configure --enable-loadable-sqlite-extensions --enable-optimizations && make -j install && rm -r /tmp/Python-3.8.6.tar.xz /tmp/Python-3.8.6"\n#7 started: 2022-09-01 03:10:07.653181139 +0000 UTC\n#7 cached: true\n#7 pulling sha256:f2e056f59b4d8bc8b431e3ac00939741d71acf23759f9faabd7afdfbabe55811\n#7 pulling sha256:9d8a655bce8b57175278c662bdde982e3b294b07cb3218ce032443a049c021e8\n#7 pulling sha256:9d8a655bce8b57175278c662bdde982e3b294b07cb3218ce032443a049c021e8 30.5s done\n#7 pulling sha256:f2e056f59b4d8bc8b431e3ac00939741d71acf23759f9faabd7afdfbabe55811 35.7s done\n#7 completed: 2022-09-01 03:10:59.068697093 +0000 UTC\n#7 duration: 51.415515954s\n\n\n#9 [4/7] COPY ./requirements.txt /\n#9 digest: sha256:8a5f26bd36a1f6cbd631d64c369591a13ec73bd44a5393ffbd764a78a2f1f77c\n#9 name: "[4/7] COPY ./requirements.txt /"\n#9 started: 2022-09-01 03:10:59.143815932 +0000 UTC\n#9 completed: 2022-09-01 03:11:39.765734829 +0000 UTC\n#9 duration: 40.621918897s\n\n\n#10 [5/7] RUN pip3 install -r /requirements.txt\n#10 digest: sha256:cf3cc96208fec69f6c0daf8898faf165439f91ad86ceace2f3d1a774559d95e6\n#10 name: "[5/7] RUN pip3 install -r /requirements.txt"\n#10 started: 2022-09-01 03:11:39.824937314 +0000 UTC\n#10 16.62 WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by \'ReadTimeoutError("HTTPSConnectionPool(host=\'pypi.org\', port=443): Read timed out. (read timeout=15)")\': /simple/alembic/\n#10 32.38 WARNING: Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by \'ReadTimeoutError("HTTPSConnectionPool(host=\'pypi.org\', port=443): Read timed out. (read timeout=15)")\': /simple/alembic/\n#10 48.64 WARNING: Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by \'ReadTimeoutError("HTTPSConnectionPool(host=\'pypi.org\', port=443): Read timed out. (read timeout=15)")\': /simple/alembic/\n#10 65.90 WARNING: Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by \'ReadTimeoutError("HTTPSConnectionPool(host=\'pypi.org\', port=443): Read timed out. (read timeout=15)")\': /simple/alembic/\n#10 85.15 WARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by \'ReadTimeoutError("HTTPSConnectionPool(host=\'pypi.org\', port=443): Read timed out. (read timeout=15)")\': /simple/alembic/\n#10 100.4 ERROR: Could not find a version that satisfies the requirement alembic==1.4.0 (from -r /requirements.txt (line 1)) (from versions: none)\n#10 100.4 ERROR: No matching distribution found for alembic==1.4.0 (from -r /requirements.txt (line 1))\n#10 completed: 2022-09-01 03:13:26.269309865 +0000 UTC\n#10 duration: 1m46.444372551s\n#10 error: "executor failed running [/bin/sh -c pip3 install -r /requirements.txt]: exit code: 1"\n\nexecutor failed running [/bin/sh -c pip3 install -r /requirements.txt]: exit code: 1\ndocker/generated.mk:100: recipe for target \'base-image\' failed\nmake: *** [base-image] Error 1\n'} ERROR:root:Error conducting experiment. Extras: {'traceback': 'Traceback (most recent call last):\n File "/work/src/experiment/dispatcher.py", line 203, in main\n dispatcher_main()\n File "/work/src/experiment/dispatcher.py", line 156, in dispatcher_main\n trials = build_images_for_trials(experiment.fuzzers, experiment.benchmarks,\n File "/work/src/experiment/dispatcher.py", line 116, in build_images_for_trials\n builder.build_base_images()\n File "/work/src/experiment/build/builder.py", line 98, in build_base_images\n return buildlib.build_base_images()\n File "/work/src/experiment/build/local_build.py", line 38, in build_base_images\n return make([\'base-image\', \'worker\'])\n File "/work/src/experiment/build/local_build.py", line 33, in make\n return new_process.execute(command, cwd=utils.ROOT_DIR)\n File "/work/src/common/new_process.py", line 124, in execute\n raise subprocess.CalledProcessError(retcode, command)\nsubprocess.CalledProcessError: Command \'[\'make\', \'-j\', \'base-image\', \'worker\']\' returned non-zero exit status 2.\n'} Traceback (most recent call last): File "/work/src/experiment/dispatcher.py", line 221, in sys.exit(main()) File "/work/src/experiment/dispatcher.py", line 206, in main raise error File "/work/src/experiment/dispatcher.py", line 203, in main dispatcher_main() File "/work/src/experiment/dispatcher.py", line 156, in dispatcher_main trials = build_images_for_trials(experiment.fuzzers, experiment.benchmarks, File "/work/src/experiment/dispatcher.py", line 116, in build_images_for_trials builder.build_base_images() File "/work/src/experiment/build/builder.py", line 98, in build_base_images return buildlib.build_base_images() File "/work/src/experiment/build/local_build.py", line 38, in build_base_images return make(['base-image', 'worker']) File "/work/src/experiment/build/local_build.py", line 33, in make return new_process.execute(command, cwd=utils.ROOT_DIR) File "/work/src/common/new_process.py", line 124, in execute raise subprocess.CalledProcessError(retcode, command) subprocess.CalledProcessError: Command '['make', '-j', 'base-image', 'worker']' returned non-zero exit status 2. ` and python version is Python 3.8.10

Any pointers to resolve this error? Thanks.

Originally posted by @amie0 in https://github.com/google/fuzzbench/issues/1211#issuecomment-1233690051

ali-soomro commented 1 year ago

Hi, did you ever get FuzzBench working?