search

HexHive / magma

A ground-truth fuzzing benchmark suite based on real programs with real bugs.
https://hexhive.epfl.ch/magma
283 stars 82 forks source link

How to evaluate my fuzzer with Magma? #171

Open JWH-96 opened 1 week ago

JWH-96 commented 1 week ago

I want to evaluate my fuzzer with Magma, but I do not have a public git repo. How can I modify the Docker image to use my local code?Could you please provide detailed implementation steps?

adrianherrera commented 1 week ago

Almost all of the fuzzers integrated into magma pull local files into the docker container during building. Eg angora has local files under the src directory and these are available inside the container under $FUZZER/src (see angoras fetch.sh). You can just put your entire code base inside a sec directory and do the same thing; no need to git clone anything.

On Tue, 10 Sep 2024 at 6:59 PM, 123huahua @.***> wrote:

I want to evaluate my fuzzer with Magma, but I do not have a public git repo. How can I modify the Docker image to use my local code?Could you please provide detailed implementation steps?

— Reply to this email directly, view it on GitHub https://github.com/HexHive/magma/issues/171, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACB2DESTNI26YBKS2GO6KJTZV2YGRAVCNFSM6AAAAABN6IKA5GVHI2DSMVQWIX3LMV43ASLTON2WKOZSGUYTKOBTHE2DSMQ . You are receiving this because you are subscribed to this thread.Message ID: @.***>

JWH-96 commented 1 week ago

How should I modify the script files in the fuzzer folder and what should I pay attention to? The script copied from other fuzzers cannot run my fuzzer properly.

JWH-96 commented 1 week ago

Because my fuzzer is implemented based on afl++, but it is different from the version of afl++ in magma. I copied those script files to my fuzzer and encountered the following error。Using the same script to run AFL++, everything is OK。

20240910202710

[20/20] RUN /magma/fuzzers/mesfuzz/instrument.sh --progress=plain --no-cache: 0.439 + CXXFLAGS='-include /magma/magma/src/canary.h -DMAGMA_ENABLE_CANARIES -DMAGMA_FATAL_CANARIES -g -O0 -stdlib=libc++' 0.439 + export OUT=/magma_out/afl 0.439 + OUT=/magma_out/afl 0.439 + export 'LDFLAGS=-L/magma_out -g -L/magma_out/afl' 0.439 + LDFLAGS='-L/magma_out -g -L/magma_out/afl' 0.439 + /magma/magma/build.sh 0.449 + MAGMA_STORAGE=/magma_shared/canaries.raw 0.449 + /magma/fuzzers/mesfuzz/repo/afl-clang-fast -include /magma/magma/src/canary.h -DMAGMA_ENABLE_CANARIES -DMAGMA_FATAL_CANARIES -g -O0 '-DMAGMA_STORAGE="/magma_shared/canaries.raw"' -c /magma/magma/src/canary.c -fPIC -I /magma/magma/src/ -o /magma_out/afl/canary.o -L/magma_out -g -L/magma_out/afl 0.456 /magma/fuzzers/mesfuzz/repo/afl-clang-fast: /lib/x86_64-linux-gnu/libc.so.6: version GLIBC_2.33' not found (required by /magma/fuzzers/mesfuzz/repo/afl-clang-fast) 0.456 /magma/fuzzers/mesfuzz/repo/afl-clang-fast: /lib/x86_64-linux-gnu/libc.so.6: versionGLIBC_2.34' not found (required by /magma/fuzzers/mesfuzz/repo/afl-clang-fast)

Dockerfile:86

84 | ENV LDFLAGS -L"${OUT}" -g 85 |
86 | >>> RUN ${FUZZER}/instrument.sh --progress=plain --no-cache 87 |
88 | ENTRYPOINT "${MAGMA}/run.sh"

ERROR: failed to solve: process "/bin/sh -c ${FUZZER}/instrument.sh --progress=plain --no-cache" did not complete successfully: exit code: 1

adrianherrera commented 1 week ago

You need to build the fuzzer in the container; don’t copy across prebuilt binaries. It looks like you have a gcc mismatch

On Tue, 10 Sep 2024 at 10:29 PM, 123huahua @.***> wrote:

Because my fuzzer is implemented based on afl++, but it is different from the version of afl++ in magma. I copied those script files to my fuzzer and encountered the following error。Using the same script to run AFL++, everything is OK。 20240910202710.png (view on web) https://github.com/user-attachments/assets/cf5b0bbb-4690-450f-bcf5-45f90dea63a0

[20/20] RUN /magma/fuzzers/mesfuzz/instrument.sh --progress=plain --no-cache: 0.439 + CXXFLAGS='-include /magma/magma/src/canary.h -DMAGMA_ENABLE_CANARIES -DMAGMA_FATAL_CANARIES -g -O0 -stdlib=libc++' 0.439 + export OUT=/magma_out/afl 0.439 + OUT=/magma_out/afl 0.439 + export 'LDFLAGS=-L/magma_out -g -L/magma_out/afl' 0.439 + LDFLAGS='-L/magma_out -g -L/magma_out/afl' 0.439 + /magma/magma/build.sh 0.449 + MAGMA_STORAGE=/magma_shared/canaries.raw 0.449 + /magma/fuzzers/mesfuzz/repo/afl-clang-fast -include /magma/magma/src/canary.h -DMAGMA_ENABLE_CANARIES -DMAGMA_FATAL_CANARIES -g -O0 '-DMAGMA_STORAGE="/magma_shared/canaries.raw"' -c /magma/magma/src/canary.c -fPIC -I /magma/magma/src/ -o /magma_out/afl/canary.o -L/magma_out -g -L/magma_out/afl 0.456 /magma/fuzzers/mesfuzz/repo/afl-clang-fast: /lib/x86_64-linux-gnu/libc.so.6: version GLIBC_2.33' not found (required by /magma/fuzzers/mesfuzz/repo/afl-clang-fast) 0.456 /magma/fuzzers/mesfuzz/repo/afl-clang-fast: /lib/x86_64-linux-gnu/libc.so.6: version GLIBC_2.34' not found (required by /magma/fuzzers/mesfuzz/repo/afl-clang-fast)

Dockerfile:86 84 | ENV LDFLAGS -L"${OUT}" -g 85 | 86 | >>> RUN ${FUZZER}/instrument.sh --progress=plain --no-cache 87 | 88 | ENTRYPOINT "${MAGMA}/run.sh"

ERROR: failed to solve: process "/bin/sh -c ${FUZZER}/instrument.sh --progress=plain --no-cache" did not complete successfully: exit code: 1

— Reply to this email directly, view it on GitHub https://github.com/HexHive/magma/issues/171#issuecomment-2340583951, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACB2DEV46DXGTHAWKH7327DZV3Q4DAVCNFSM6AAAAABN6IKA5GVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGNBQGU4DGOJVGE . You are receiving this because you commented.Message ID: @.***>

JWH-96 commented 6 days ago

Why does the fuzzing process end in just a few seconds when the TIMEOUT is set to 1h?

1 2 3

adrianherrera commented 6 days ago

It is impossible to debug this based on the information given. I would look through the magma logs produced in the out directory and see if you can find any errors. I suspect the fuzzer is crashing.

On Wed, 11 Sep 2024 at 10:54 PM, 123huahua @.***> wrote:

Why does the fuzzing process end in just a few seconds when the TIMEOUT is set to 1h?

1.png (view on web) https://github.com/user-attachments/assets/6543b868-6fc4-4fbd-a1ab-414094ccd865 2.png (view on web) https://github.com/user-attachments/assets/5a413ab7-5bb4-4fdd-8e8c-63d6c9ee4f97 3.png (view on web) https://github.com/user-attachments/assets/fa5bbf23-d3c6-4ad8-ba44-62439c496b13

— Reply to this email directly, view it on GitHub https://github.com/HexHive/magma/issues/171#issuecomment-2343590351, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACB2DES4K7LDUK6WJBBTRIDZWA4PPAVCNFSM6AAAAABN6IKA5GVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGNBTGU4TAMZVGE . You are receiving this because you commented.Message ID: @.***>