google / fuzzbench

FuzzBench - Fuzzer benchmarking as a service.
https://google.github.io/fuzzbench/
Apache License 2.0
1.09k stars 266 forks source link

No snapshot data when running local experiment #1948

Closed harrison4ride closed 8 months ago

harrison4ride commented 8 months ago

Hi all,

I have successfully built images for aflplusplus, libxml, and run local experiment using

PYTHONPATH=. python3 experiment/run_experiment.py \
--experiment-config experiment-config.yaml \
--benchmarks libxml2_xml \
--experiment-name test1 \
--fuzzers aflplusplus ```

My experiment-config.yaml file:
```# The number of trials of a fuzzer-benchmark pair.
trials: 1

# The amount of time in seconds that each trial is run for.
# 1 day = 24 * 60 * 60 = 86400
max_total_time: 100

snapshot_period: 10

# The location of the docker registry.
# FIXME: Support custom docker registry.
# See https://github.com/google/fuzzbench/issues/777
docker_registry: gcr.io/fuzzbench

# The local experiment folder that will store most of the experiment data.
# Please use an absolute path.
experiment_filestore: /home/hxxzhang/tmp/experiment-data

# The local report folder where HTML reports and summary data will be stored.
# Please use an absolute path.
report_filestore: /home/hxxzhang/tmp/report-data

# Flag that indicates this is a local experiment.
local_experiment: true

Then the output just repeats


INFO:root:Reading experiment data from db.                                                                                                                    
INFO:root:Done reading experiment data from db.                                                                                                               
WARNING:root:No snapshot data.                                                                                                                                
INFO:root:Finding trials to schedule.                                          
INFO:root:Starting trials.                                                                                                                                    
INFO:root:Started 0 trials.                                                    
INFO:root:Reading experiment data from db.                                                                                                                    
INFO:root:Done reading experiment data from db.                                                                                                               
WARNING:root:No snapshot data.                                                                                                                                
INFO:root:Finding trials to schedule.                                                                                                                         
INFO:root:Starting trials.                                                                                                                                    
INFO:root:Started 0 trials.
INFO:root:Measuring all trials.
INFO:root:Measuring all trials.
INFO:root:Reading experiment data from db.
INFO:root:Done reading experiment data from db.
WARNING:root:No snapshot data.```
I checked my local.db there is no data in snapshot table. I have been stuck in this place for 1 week, any clues could be helpful. PLS!
DataCorrupted commented 8 months ago

Please try modify experiment_filestore from /home/hxxzhang/tmp/... to /tmp/.... Same for report_filestore

ali-soomro commented 7 months ago

I'm having the same issue, my yaml file is:

# The number of trials of a fuzzer-benchmark pair.
trials: 1

# The amount of time in seconds that each trial is run for.
# 1 day = 24 * 60 * 60 = 86400
max_total_time: 1800

# The location of the docker registry.
# FIXME: Support custom docker registry.
# See https://github.com/google/fuzzbench/issues/777
docker_registry: gcr.io/fuzzbench

# The local experiment folder that will store most of the experiment data.
# Please use an absolute path.
experiment_filestore: /tmp/experiment-data

# The local report folder where HTML reports and summary data will be stored.
# Please use an absolute path.
report_filestore: /tmp/report-data

# Flag that indicates this is a local experiment.
local_experiment: true
DataCorrupted commented 7 months ago

Do you have existing /tmp/experiment-data?

ali-soomro commented 7 months ago

Do you have existing /tmp/experiment-data?

Yes

From the looks of it the binary, source and experiment.yaml are successfully writing to this. However, no coverage information

DataCorrupted commented 7 months ago

I don't think you should.

On Tue, Feb 6, 2024, 5:42 PM ali-soomro @.***> wrote:

Do you have existing /tmp/experiment-data?

Yes

— Reply to this email directly, view it on GitHub https://github.com/google/fuzzbench/issues/1948#issuecomment-1931097033, or unsubscribe https://github.com/notifications/unsubscribe-auth/AFQQFGCLSOJBKR4LKRXM2CLYSLLWZAVCNFSM6AAAAABCEWA6DGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMZRGA4TOMBTGM . You are receiving this because you commented.Message ID: @.***>

ali-soomro commented 7 months ago

I don't think you should. On Tue, Feb 6, 2024, 5:42 PM ali-soomro @.> wrote: Do you have existing /tmp/experiment-data? Yes — Reply to this email directly, view it on GitHub <#1948 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AFQQFGCLSOJBKR4LKRXM2CLYSLLWZAVCNFSM6AAAAABCEWA6DGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMZRGA4TOMBTGM . You are receiving this because you commented.Message ID: @.>

Sorry, just to clarify, I had tried an earlier suggestion of changing my experiment filestore and report data in the experiment config yml as follows:

I changed my experiment_filestore from /home/berry/experiment-data to /tmp/experiment-data Likewise, my report_filestore went from /home/berry/report-data to /tmp/report-data.

After this, I was running a local experiment, and as such, I had this directory under /tmp/.

I no longer have these directories