Can you provide scripts or instructions for artifact evaluation? + Docker image?

[DaramG]

I'm trying to reproduce your results with your code. But, DockerFile seems to depend on some network environment. It does not work in my machine. Do you have plan to update Docker image also? And can you provide scripts or instructions for artifact evaluation?

[cyruscyliu]

Could you please copy and paste the error you had? Then, I can fix the problem respectively. For the artifact evaluation, do you want to run the fuzzer only or collect the coverage? More information would be nicer!

[DaramG]

Thanks for your reply!

Here are a part of errors when I executed make qemu in videzzo inside the Docker

~/videzzo/videzzo_qemu/qemu ~/videzzo/videzzo_qemu
+ mkdir out-san
mkdir: cannot create directory 'out-san': File exists
+ pushd out-san
~/videzzo/videzzo_qemu/qemu/out-san ~/videzzo/videzzo_qemu/qemu ~/videzzo/videzzo_qemu
+ CC=clang
+ CXX=clang++
+ ../configure --enable-videzzo --enable-fuzzing --enable-debug --disable-werror --enable-sanitizers --enable-spice '--target-list=i386-softmmu x86_64-softmmu arm-softmmu aarch64-softmmu'
ERROR: unknown option --enable-videzzo
Try '../configure --help' for more information
+ ninja qemu-videzzo-i386 qemu-videzzo-x86_64 qemu-videzzo-arm qemu-videzzo-aarch64
ninja: error: loading 'build.ninja': No such file or directory
+ popd
~/videzzo/videzzo_qemu/qemu ~/videzzo/videzzo_qemu
+ popd
~/videzzo/videzzo_qemu
make[2]: Leaving directory '/root/videzzo/videzzo_qemu'
bash -x 0004-zip-qemu-targets.sh san
+ CONTROL=san
+ DEST_DIR=/root/videzzo/videzzo_qemu/out-san
+ mkdir /root/videzzo/videzzo_qemu/out-san
mkdir: cannot create directory '/root/videzzo/videzzo_qemu/out-san': File exists
+ pushd qemu/out-san
~/videzzo/videzzo_qemu/qemu/out-san ~/videzzo/videzzo_qemu
+ cp -r ../pc-bios /root/videzzo/videzzo_qemu/out-san/pc-bios
+ archs=(i386 x86_64 arm aarch64)
+ for arch in ${archs[@]}
++ ./qemu-videzzo-i386
0004-zip-qemu-targets.sh: line 15: ./qemu-videzzo-i386: No such file or directory
++ awk '$1 ~ /\*/  {print $2}'
+ targets=
++ echo ''
++ head -n -1
+ for arch in ${archs[@]}
++ ./qemu-videzzo-x86_64
0004-zip-qemu-targets.sh: line 15: ./qemu-videzzo-x86_64: No such file or directory
++ awk '$1 ~ /\*/  {print $2}'
+ targets=
++ echo ''
++ head -n -1
+ for arch in ${archs[@]}
++ ./qemu-videzzo-arm
0004-zip-qemu-targets.sh: line 15: ./qemu-videzzo-arm: No such file or directory
++ awk '$1 ~ /\*/  {print $2}'
+ targets=
++ echo ''
++ head -n -1
+ for arch in ${archs[@]}
++ ./qemu-videzzo-aarch64
0004-zip-qemu-targets.sh: line 15: ./qemu-videzzo-aarch64: No such file or directory
++ awk '$1 ~ /\*/  {print $2}'
+ targets=
++ echo ''
++ head -n -1
+ popd
~/videzzo/videzzo_qemu
+ pushd /root/videzzo/videzzo_qemu/out-san
~/videzzo/videzzo_qemu/out-san ~/videzzo/videzzo_qemu
+ popd
~/videzzo/videzzo_qemu
make[1]: Leaving directory '/root/videzzo/videzzo_qemu'

For the artifact evaluation, we want both modes, running fuzzer only and collecting coverage.

Thanks!

[cyruscyliu]

Thank you. Let me re-check the Dockerfile today.

[cyruscyliu]

Yeah, I guess the Dockerfile has some problems with the toolchain and the instructions for artifact evaluation are not clear. I will try to update them today ...

[cyruscyliu]

Hi, could you test my latest commit? It works on my side.

[cyruscyliu]

For any large evaluation, you might want to have a look at my evaluation code against old QEMU.

[DaramG]

It works now! Thanks for your help!!