Open dliyanage opened 3 years ago
Thank you for reporting this issue and sorry for the delay. I can't reproduce this issue locally so it seems to be an issue with your setup. Looking at the logs you posted, this part seems important:
buildkit not supported by daemon\ndocker/generated.mk:115: recipe for target 'worker' failed\nmake: *** [worker] Error 1\n"}
It seems like your setup doesn't support docker buildkit. FuzzBench depends on this. What version of docker are you using? This seems to work with docker 20. Maybe you need to enable buildkit? This might be necessary, i'll add this to the docs.
What is nectar server by the way? Never heard of this and having trouble finding out.
hi I cant run it locally make presubmit was successful.Then I wanted to run the experiment for multiple trials and I tried it by placing the experiment-config.yaml file. Then run the command: `PYTHONPATH=. python3 experiment/run_experiment.py \
--experiment-config experiment-config.yaml \ --benchmarks freetype2-2017 bloaty_fuzz_target \ --experiment-name testexperiment \ --fuzzers afl libfuzzer
unfortunatelly
INFO:root:Starting experiment. ERROR:root:Executed command: "make -j base-image worker" returned: 2. Extras: {'output': 'arted\n Running setup.py install for importlab: finished with status \'done\'\n Running setup.py install for serverfiles: started\n Running setup.py install for serverfiles: finished with status \'done\'\n Running setup.py install for mozinfo: started\n Running setup.py install for mozinfo: finished with status \'done\'\n Running setup.py install for alembic: started\n Running setup.py install for alembic: finished with status \'done\'\n Running setup.py install for google-cloud-profiler: started\n Running setup.py install for google-cloud-profiler: finished with status \'done\'\nSuccessfully installed AnyQt-0.2.0 Jinja2-2.11.3 Mako-1.2.2 MarkupSafe-2.0.1 Orange3-3.28.0 PyQt5-5.15.7 PyQt5-Qt5-5.15.2 PyQt5-sip-12.11.0 PyQtWebEngine-5.15.6 PyQtWebEngine-Qt5-5.15.2 PyYAML-5.4 SecretStorage-3.3.3 alembic-1.4.0 astroid-2.6.6 attrs-22.1.0 baycomp-1.0.2 bottleneck-1.3.5 cachecontrol-0.12.11 cachetools-4.2.4 certifi-2022.6.15 cffi-1.15.1 chardet-5.0.0 charset-normalizer-2.1.1 click-8.1.3 clusterfuzz-0.0.1a0 commonmark-0.9.1 cryptography-37.0.4 cycler-0.11.0 dictdiffer-0.9.0 distro-1.7.0 docutils-0.19 et-xmlfile-1.1.0 fonttools-4.37.1 google-api-core-1.32.0 google-api-python-client-2.5.0 google-auth-1.30.1 google-auth-httplib2-0.1.0 google-auth-oauthlib-0.5.2 google-cloud-core-1.7.3 google-cloud-datastore-1.15.5 google-cloud-error-reporting-1.1.2 google-cloud-logging-1.15.1 google-cloud-monitoring-2.11.1 google-cloud-ndb-1.11.1 google-cloud-profiler-4.0.0 google-cloud-secret-manager-2.4.0 google-cloud-storage-2.2.1 google-crc32c-1.3.0 google-resumable-media-2.3.3 googleapis-common-protos-1.56.4 grpc-google-iam-v1-0.12.4 grpcio-1.47.0 h11-0.13.0 httpcore-0.12.3 httplib2-0.20.4 httpx-0.16.1 idna-3.3 importlab-0.7 importlib-metadata-4.12.0 iniconfig-1.1.1 isort-5.10.1 jeepney-0.8.0 joblib-1.1.0 keyring-23.8.2 keyrings.alt-4.1.1 kiwisolver-1.4.4 lazy-object-proxy-1.7.1 libcst-0.4.7 lockfile-0.12.2 matplotlib-3.5.3 mccabe-0.6.1 mozfile-2.1.0 mozinfo-1.2.2 mozprocess-1.3.0 msgpack-1.0.4 mypy-extensions-0.4.3 networkx-2.8.6 ninja-1.10.2.3 numpy-1.21.0 oauth2client-4.1.3 oauthlib-3.2.0 openTSNE-0.6.2 openpyxl-3.0.10 orange-canvas-core-0.1.27 orange-widget-base-4.17.0 packaging-21.3 pandas-1.2.4 patsy-0.5.2 pillow-9.2.0 pluggy-0.13.1 proto-plus-1.22.1 protobuf-3.20.1 psutil-5.9.0 psycopg2-binary-2.8.4 py-1.11.0 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pyfakefs-3.7.1 pylint-2.7.4 pymemcache-3.5.2 pyparsing-3.0.9 pyqtgraph-0.12.4 pytest-6.1.2 python-dateutil-2.8.1 python-editor-1.0.4 python-louvain-0.16 pytype-2021.4.15 pytz-2019.3 qasync-0.23.0 redis-3.5.3 requests-2.28.1 requests-oauthlib-1.3.1 rfc3986-1.5.0 rq-1.4.3 rsa-4.9 scikit-learn-1.1.2 scikit-posthocs-0.6.2 scipy-1.6.2 seaborn-0.11.1 serverfiles-0.3.1 six-1.16.0 sniffio-1.2.0 sqlalchemy-1.3.19 statsmodels-0.13.2 threadpoolctl-3.1.0 toml-0.10.2 typed-ast-1.5.4 typing-extensions-4.3.0 typing-inspect-0.8.0 uritemplate-3.0.1 urllib3-1.26.12 wrapt-1.12.1 xlrd-2.0.1 xlsxwriter-3.0.3 yapf-0.30.0 zipp-3.8.1\nsource .venv/bin/activate && PYTHONPATH=. python3 docker/generate_makefile.py docker/generated.mk\ndocker pull ubuntu:xenial\nxenial: Pulling from library/ubuntu\nDigest: sha256:91bd29a464fdabfcf44e29e1f2a5f213c6dfa750b6290e40dd6998ac79da3c41\nStatus: Image is up to date for ubuntu:xenial\ndocker build \\n--tag gcr.io/fuzzbench/base-image \\n--build-arg BUILDKIT_INLINE_CACHE=1 \\n--cache-from gcr.io/fuzzbench/base-image \\n--file docker/base-image/Dockerfile \\n.\n\n#2 [internal] load .dockerignore\n#2 digest: sha256:16d0df9858d81eca73975e6c46a8b6ead0852f543011686baa9afee625d2b71c\n#2 name: "[internal] load .dockerignore"\n#2 started: 2022-09-01 03:09:54.577877083 +0000 UTC\n#2 completed: 2022-09-01 03:09:54.577918759 +0000 UTC\n#2 duration: 41.676µs\n#2 started: 2022-09-01 03:09:54.57821986 +0000 UTC\n#1 ...\n\n#1 [internal] load build definition from Dockerfile\n#1 digest: sha256:57433ba08e23227e9d39aedbe741164987f5e7d74e067286c5ebc39de04a048a\n#1 name: "[internal] load build definition from Dockerfile"\n#1 started: 2022-09-01 03:09:54.536909796 +0000 UTC\n#1 completed: 2022-09-01 03:09:54.536962975 +0000 UTC\n#1 duration: 53.179µs\n#1 started: 2022-09-01 03:09:54.537161007 +0000 UTC\n#1 completed: 2022-09-01 03:09:58.688779892 +0000 UTC\n#1 duration: 4.151618885s\n#1 transferring dockerfile: 2.00kB done\n\n\n#2 [internal] load .dockerignore\n#2 transferring context: 143B done\n#2 completed: 2022-09-01 03:10:00.139782363 +0000 UTC\n#2 duration: 5.561562503s\n\n\n#5 [1/7] FROM docker.io/library/ubuntu:xenial\n#5 digest: sha256:14dd403f25ae9e519bd44f27dc17af7ce80f5a4f9f48ca88108364e67ec5d360\n#5 name: "[1/7] FROM docker.io/library/ubuntu:xenial"\n#5 started: 2022-09-01 03:10:00.658333095 +0000 UTC\n#5 completed: 2022-09-01 03:10:00.659016816 +0000 UTC\n#5 duration: 683.721µs\n\n\n#8 [internal] load build context\n#8 digest: sha256:cfa463da6dbd3e62ce7ad89a34dfe2db0a41eff97ee883e6df3081c8981aa9cf\n#8 name: "[internal] load build context"\n#8 started: 2022-09-01 03:10:00.658403926 +0000 UTC\n#8 completed: 2022-09-01 03:10:00.658551012 +0000 UTC\n#8 duration: 147.086µs\n\n\n#3 [internal] load metadata for docker.io/library/ubuntu:xenial\n#3 digest: sha256:f62bae18ed588840f045c881b2502661f2a89578a2f8f99b83f5200ee83dd25b\n#3 name: "[internal] load metadata for docker.io/library/ubuntu:xenial"\n#3 started: 2022-09-01 03:10:00.655364511 +0000 UTC\n#3 completed: 2022-09-01 03:10:00.655844253 +0000 UTC\n#3 duration: 479.742µs\n\n\n#4 importing cache manifest from gcr.io/fuzzbench/base-image\n#4 digest: sha256:9b220ebb4d4daceed1b96df9b6dca076e762f1cd612574f010b82d5b9e5ad896\n#4 name: "importing cache manifest from gcr.io/fuzzbench/base-image"\n#4 started: 2022-09-01 03:10:00.657580785 +0000 UTC\n#4 completed: 2022-09-01 03:10:03.905354032 +0000 UTC\n#4 duration: 3.247773247s\n\n\n#8 [internal] load build context\n#8 started: 2022-09-01 03:10:03.906843654 +0000 UTC\n#8 transferring context:\n#8 transferring context: 608B done\n#8 completed: 2022-09-01 03:10:07.458310793 +0000 UTC\n#8 duration: 3.551467139s\n\n\n#6 [2/7] RUN apt-get update -y && apt-get install -y build-essential ...\n#6 digest: sha256:3a1b0960f17eed529eaae93dc8fd3554f2743dd584d8fa569cdbe6bc734371ac\n#6 name: "[2/7] RUN apt-get update -y && apt-get install -y build-essential rsync curl zlib1g-dev libncurses5-dev libgdbm-dev libnss3-dev libssl-dev libreadline-dev libffi-dev virtualenv libbz2-dev liblzma-dev libsqlite3-dev"\n#6 started: 2022-09-01 03:10:07.653181139 +0000 UTC\n#6 completed: 2022-09-01 03:10:07.653181139 +0000 UTC\n#6 duration: 0s\n#6 cached: true\n\n\n#7 [3/7] RUN cd /tmp/ && curl -O https://www.python.org/ftp/python/3.8....\n#7 digest: sha256:c5b9d1fb3371e84dbc60b76e7b2f8d5913bc93bcd4910ab65daa03e7b2ec4bb0\n#7 name: "[3/7] RUN cd /tmp/ && curl -O https://www.python.org/ftp/python/3.8.6/Python-3.8.6.tar.xz && tar -xvf Python-3.8.6.tar.xz && cd Python-3.8.6 && ./configure --enable-loadable-sqlite-extensions --enable-optimizations && make -j install && rm -r /tmp/Python-3.8.6.tar.xz /tmp/Python-3.8.6"\n#7 started: 2022-09-01 03:10:07.653181139 +0000 UTC\n#7 cached: true\n#7 pulling sha256:f2e056f59b4d8bc8b431e3ac00939741d71acf23759f9faabd7afdfbabe55811\n#7 pulling sha256:9d8a655bce8b57175278c662bdde982e3b294b07cb3218ce032443a049c021e8\n#7 pulling sha256:9d8a655bce8b57175278c662bdde982e3b294b07cb3218ce032443a049c021e8 30.5s done\n#7 pulling sha256:f2e056f59b4d8bc8b431e3ac00939741d71acf23759f9faabd7afdfbabe55811 35.7s done\n#7 completed: 2022-09-01 03:10:59.068697093 +0000 UTC\n#7 duration: 51.415515954s\n\n\n#9 [4/7] COPY ./requirements.txt /\n#9 digest: sha256:8a5f26bd36a1f6cbd631d64c369591a13ec73bd44a5393ffbd764a78a2f1f77c\n#9 name: "[4/7] COPY ./requirements.txt /"\n#9 started: 2022-09-01 03:10:59.143815932 +0000 UTC\n#9 completed: 2022-09-01 03:11:39.765734829 +0000 UTC\n#9 duration: 40.621918897s\n\n\n#10 [5/7] RUN pip3 install -r /requirements.txt\n#10 digest: sha256:cf3cc96208fec69f6c0daf8898faf165439f91ad86ceace2f3d1a774559d95e6\n#10 name: "[5/7] RUN pip3 install -r /requirements.txt"\n#10 started: 2022-09-01 03:11:39.824937314 +0000 UTC\n#10 16.62 WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by \'ReadTimeoutError("HTTPSConnectionPool(host=\'pypi.org\', port=443): Read timed out. (read timeout=15)")\': /simple/alembic/\n#10 32.38 WARNING: Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by \'ReadTimeoutError("HTTPSConnectionPool(host=\'pypi.org\', port=443): Read timed out. (read timeout=15)")\': /simple/alembic/\n#10 48.64 WARNING: Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by \'ReadTimeoutError("HTTPSConnectionPool(host=\'pypi.org\', port=443): Read timed out. (read timeout=15)")\': /simple/alembic/\n#10 65.90 WARNING: Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by \'ReadTimeoutError("HTTPSConnectionPool(host=\'pypi.org\', port=443): Read timed out. (read timeout=15)")\': /simple/alembic/\n#10 85.15 WARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by \'ReadTimeoutError("HTTPSConnectionPool(host=\'pypi.org\', port=443): Read timed out. (read timeout=15)")\': /simple/alembic/\n#10 100.4 ERROR: Could not find a version that satisfies the requirement alembic==1.4.0 (from -r /requirements.txt (line 1)) (from versions: none)\n#10 100.4 ERROR: No matching distribution found for alembic==1.4.0 (from -r /requirements.txt (line 1))\n#10 completed: 2022-09-01 03:13:26.269309865 +0000 UTC\n#10 duration: 1m46.444372551s\n#10 error: "executor failed running [/bin/sh -c pip3 install -r /requirements.txt]: exit code: 1"\n\nexecutor failed running [/bin/sh -c pip3 install -r /requirements.txt]: exit code: 1\ndocker/generated.mk:100: recipe for target \'base-image\' failed\nmake: *** [base-image] Error 1\n'} ERROR:root:Error conducting experiment. Extras: {'traceback': 'Traceback (most recent call last):\n File "/work/src/experiment/dispatcher.py", line 203, in main\n dispatcher_main()\n File "/work/src/experiment/dispatcher.py", line 156, in dispatcher_main\n trials = build_images_for_trials(experiment.fuzzers, experiment.benchmarks,\n File "/work/src/experiment/dispatcher.py", line 116, in build_images_for_trials\n builder.build_base_images()\n File "/work/src/experiment/build/builder.py", line 98, in build_base_images\n return buildlib.build_base_images()\n File "/work/src/experiment/build/local_build.py", line 38, in build_base_images\n return make([\'base-image\', \'worker\'])\n File "/work/src/experiment/build/local_build.py", line 33, in make\n return new_process.execute(command, cwd=utils.ROOT_DIR)\n File "/work/src/common/new_process.py", line 124, in execute\n raise subprocess.CalledProcessError(retcode, command)\nsubprocess.CalledProcessError: Command \'[\'make\', \'-j\', \'base-image\', \'worker\']\' returned non-zero exit status 2.\n'} Traceback (most recent call last): File "/work/src/experiment/dispatcher.py", line 221, insys.exit(main()) File "/work/src/experiment/dispatcher.py", line 206, in main raise error File "/work/src/experiment/dispatcher.py", line 203, in main dispatcher_main() File "/work/src/experiment/dispatcher.py", line 156, in dispatcher_main trials = build_images_for_trials(experiment.fuzzers, experiment.benchmarks, File "/work/src/experiment/dispatcher.py", line 116, in build_images_for_trials builder.build_base_images() File "/work/src/experiment/build/builder.py", line 98, in build_base_images return buildlib.build_base_images() File "/work/src/experiment/build/local_build.py", line 38, in build_base_images return make(['base-image', 'worker']) File "/work/src/experiment/build/local_build.py", line 33, in make return new_process.execute(command, cwd=utils.ROOT_DIR) File "/work/src/common/new_process.py", line 124, in execute raise subprocess.CalledProcessError(retcode, command) subprocess.CalledProcessError: Command '['make', '-j', 'base-image', 'worker']' returned non-zero exit status 2. ` and python version is Python 3.8.10
Any pointers to resolve this error? Thanks.
I have set up a fuzzbench on a Nectar Server and integrated my AFL version (tested separately) to run on several benchmarks locally.
make presubmit
was successful.When I trigger fuzzing executions manually using say
make run-afl-freetype2-2017
, it works quite well.Then I wanted to run the experiment for multiple trials and I tried it by placing the
experiment-config.yaml
file. The following command was run thereafter.PYTHONPATH=. python3 experiment/run_experiment.py --experiment-config experiment-config.yaml --benchmarks freetype2-2017 bloaty_fuzz_target --experiment-name experiment-1 --fuzzers afl
It was showing that the experiment was starting. i.e.
INFO:root:Starting experiment.
Unfortunately, the following error popped up.
Any pointers to resolve this error? Thanks.