Is it possible to migrate a java or python benchmark from OSS-Fuzz to fuzzbench?

I am hoping to measure some java or python benchmarks with fuzzbench. So I migrate a java library (java-xmlbuilder) from OSS-Fuzz by copying Dockerfile, build.sh and XmlBuilderFuzzer.java. Then I try to test it on fuzzbench using the following steps:

make -j builder-libfuzzer-java-xmlbuilder_xmlbuilderfuzzer 
make -j run-libfuzzer-java-xmlbuilder_xmlbuilderfuzzer

But an error occured at final step:

docker run \
--cpus=1 \
--shm-size=2g \
--cap-add SYS_NICE \
--cap-add SYS_PTRACE \
-e FUZZ_OUTSIDE_EXPERIMENT=1 \
-e FORCE_LOCAL=1 \
-e TRIAL_ID=1 \
-e FUZZER=libfuzzer \
-e BENCHMARK=java-xmlbuilder_xmlbuilderfuzzer \
-e FUZZ_TARGET=XmlBuilderFuzzer \
-it gcr.io/fuzzbench/runners/libfuzzer/java-xmlbuilder_xmlbuilderfuzzer
INFO:root:Doing trial as part of experiment.
INFO:root:Starting trial.
WARNING:root:max_total_time is None. Fuzzing indefinitely.
[run_fuzzer] Running command: /out/XmlBuilderFuzzer -print_final_stats=1 -close_fd_mask=3 -fork=1 -ignore_ooms=1 -ignore_timeouts=1 -ignore_crashes=1 -entropic=1 -keep_seed=1 -cross_over_uniform_dist=1 -entropic_scale_per_exec_time=1 -detect_leaks=0 -artifact_prefix=/out/corpus/crashes/ -dict=/out/XmlBuilderFuzzer.dict /out/corpus/corpus /out/seeds
/out/XmlBuilderFuzzer: line 12: /out/jazzer_driver: No such file or directory
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/src/fuzzers/libfuzzer/fuzzer.py", line 41, in fuzz
    run_fuzzer(input_corpus, output_corpus, target_binary)
  File "/src/fuzzers/libfuzzer/fuzzer.py", line 102, in run_fuzzer
    subprocess.check_call(command)
  File "/usr/local/lib/python3.10/subprocess.py", line 369, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/out/XmlBuilderFuzzer', '-print_final_stats=1', '-close_fd_mask=3', '-fork=1', '-ignore_ooms=1', '-ignore_timeouts=1', '-ignore_crashes=1', '-entropic=1', '-keep_seed=1', '-cross_over_uniform_dist=1', '-entropic_scale_per_exec_time=1', '-detect_leaks=0', '-artifact_prefix=/out/corpus/crashes/', '-dict=/out/XmlBuilderFuzzer.dict', '/out/corpus/corpus', '/out/seeds']' returned non-zero exit status 127.
ERROR:root:Executed command: "nice -n 5 python3 -u -c from fuzzers.libfuzzer import fuzzer; fuzzer.fuzz("/out/seeds", "/out/corpus", "/out/XmlBuilderFuzzer")" returned: 1.
ERROR:root:Fuzz process returned nonzero. Extras: {'traceback': 'Traceback (most recent call last):\n  File "/src/experiment/runner.py", line 200, in run_fuzzer\n    new_process.execute(command,\n  File "/src/common/new_process.py", line 125, in execute\n    raise subprocess.CalledProcessError(retcode, command)\nsubprocess.CalledProcessError: Command \'[\'nice\', \'-n\', \'5\', \'python3\', \'-u\', \'-c\', \'from fuzzers.libfuzzer import fuzzer; fuzzer.fuzz("/out/seeds", "/out/corpus", "/out/XmlBuilderFuzzer")\']\' returned non-zero exit status 1.\n'}
INFO:root:Doing final sync.
make: *** [docker/generated.mk:275779: run-libfuzzer-java-xmlbuilder_xmlbuilderfuzzer] Error 1

Maybe the missing jazzer_driver is the key to this problem? When I test java-xmlbuilder on OSS-Fuzz, it works fine and the jazzer_fuzzer is in the /out directory. checking the /out directory of both images，the benchmark-image of fuzzbench is missing everything of jazzer:

# OSS-Fuzz /out
XmlBuilderFuzzer        XmlBuilderFuzzer.dict                   java-xmlbuilder.jar      jazzer_driver                 llvm-symbolizer
XmlBuilderFuzzer.class  XmlBuilderFuzzer_libfuzzer_address_out  jazzer_agent_deploy.jar  jazzer_driver_with_sanitizer
# fuzzbench /out
XmlBuilderFuzzer  XmlBuilderFuzzer.class  XmlBuilderFuzzer.dict  java-xmlbuilder.jar

I am not familiar with the image building process of oss-fuzz and fuzzbench. So I assuming the difference of it causing this problem. Is there any way to support java and python benchmark on fuzzbench? Any help would be appreciated...

The composite of the added benchmark java-xmlbuilder_xmlbuilderfuzzer as follow

Dockfile ``` FROM gcr.io/oss-fuzz-base/base-builder-jvm RUN sed -i s@/archive.ubuntu.com/@/mirrors.aliyun.com/@g /etc/apt/sources.list RUN curl -L https://archive.apache.org/dist/maven/maven-3/3.6.3/binaries/apache-maven-3.6.3-bin.zip \ -o maven.zip && \ unzip maven.zip -d $SRC/maven && \ rm maven.zip ENV MVN $SRC/maven/apache-maven-3.6.3/bin/mvn RUN git clone --depth 1 https://github.com/google/AFL afl RUN git clone --depth 1 https://github.com/jmurty/java-xmlbuilder java-xmlbuilder COPY build.sh *.java $SRC/ WORKDIR $SRC/java-xmlbuilder ``` build.sh ``` $MVN clean package -Dmaven.javadoc.skip=true -DskipTests=true -Dpmd.skip=true \ -Dencoding=UTF-8 -Dmaven.antrun.skip=true -Dcheckstyle.skip=true \ -Denforcer.fail=false org.apache.maven.plugins:maven-shade-plugin:3.2.4:shade CURRENT_VERSION=$($MVN org.apache.maven.plugins:maven-help-plugin:3.2.0:evaluate \ -Dexpression=project.version -q -DforceStdout) cp "target/java-xmlbuilder-$CURRENT_VERSION.jar" $OUT/java-xmlbuilder.jar ALL_JARS="java-xmlbuilder.jar" # The classpath at build-time includes the project jars in $OUT as well as the # Jazzer API. BUILD_CLASSPATH=$(echo $ALL_JARS | xargs printf -- "$OUT/%s:"):$JAZZER_API_PATH # All .jar and .class files lie in the same directory as the fuzzer at runtime. RUNTIME_CLASSPATH=$(echo $ALL_JARS | xargs printf -- "\$this_dir/%s:"):\$this_dir for fuzzer in $(find $SRC -name '*Fuzzer.java') do fuzzer_basename=$(basename -s .java $fuzzer) javac -cp $BUILD_CLASSPATH $fuzzer cp $SRC/$fuzzer_basename.class $OUT/ # Create an execution wrapper that executes Jazzer with the correct arguments. echo "#!/bin/bash # LLVMFuzzerTestOneInput for fuzzer detection. this_dir=\$(dirname "\$0") if [[ "\$@" =~ (^| )-runs=[0-9]+($| ) ]] then mem_settings='-Xmx1900m:-Xss900k' else mem_settings='-Xmx2048m:-Xss1024k' fi LD_LIBRARY_PATH="$JVM_LD_LIBRARY_PATH":\$this_dir \ \$this_dir/jazzer_driver \ --agent_path=\$this_dir/jazzer_agent_deploy.jar \ --cp=$RUNTIME_CLASSPATH \ --target_class=$fuzzer_basename \ --jvm_args="\$mem_settings" \ \$@" > $OUT/$fuzzer_basename chmod u+x $OUT/$fuzzer_basename done cp $SRC/afl/dictionaries/xml.dict $OUT/XmlBuilderFuzzer.dict ``` benchmark.yaml ``` fuzz_target: XmlBuilderFuzzer project: java-xmlbuilder ``` XmlBuilderFuzzer.java ``` import com.code_intelligence.jazzer.api.FuzzedDataProvider; import com.jamesmurty.utils.XMLBuilder; import javax.xml.parsers.ParserConfigurationException; import javax.xml.xpath.XPathExpressionException; import org.w3c.dom.DOMException; public class XmlBuilderFuzzer { public static void fuzzerTestOneInput(FuzzedDataProvider data) { try { int[] choices = data.consumeInts(data.consumeInt(1, 10)); XMLBuilder builder = XMLBuilder.create(data.consumeString(data.remainingBytes() / 2)); for (Integer choice : choices) { switch (choice % 13) { case 0: builder = builder.stripWhitespaceOnlyTextNodes(); break; case 1: builder = builder.up(data.consumeInt()); break; case 2: builder = builder.elem(data.consumeRemainingAsString()); break; case 3: builder = builder.elementBefore(data.consumeRemainingAsString()); break; case 4: builder = builder.attr( data.consumeString(data.remainingBytes() / 2), data.consumeRemainingAsString()); break; case 5: builder = builder.text(data.consumeRemainingAsString()); break; case 6: builder = builder.data(data.consumeRemainingAsString()); break; case 7: builder = builder.data(data.consumeRemainingAsBytes()); break; case 8: builder = builder.cmnt(data.consumeRemainingAsString()); break; case 9: builder = builder.inst( data.consumeString(data.remainingBytes() / 2), data.consumeRemainingAsString()); break; case 10: builder = builder.insertInstruction( data.consumeString(data.remainingBytes() / 2), data.consumeRemainingAsString()); break; case 11: builder = builder.ref(data.consumeRemainingAsString()); break; case 12: builder = builder.ns(data.consumeRemainingAsString()); break; } } } catch (ParserConfigurationException | XPathExpressionException | DOMException | IllegalStateException e) { // Known exception } } } ```

google / fuzzbench

Is it possible to migrate a java or python benchmark from OSS-Fuzz to fuzzbench? #1996