search

akhikolla / RcppDeepState

RcppDeepState, a simple way to fuzz test code in Rcpp packages
https://akhikolla.github.io./
8 stars 2 forks source link

update readme example? #69

Open tdhock opened 3 years ago

tdhock commented 3 years ago

I ran the example on the README and I got the following, which is different from what you show on the README.

> pkg <- "RcppDeepState/inst/testpkgs/testSAN"
> fun_path <- file.path(pkg, "inst/testfiles/rcpp_write_index_outofbound")
> seed_analyze<-deepstate_fuzz_fun_analyze(fun_path ,1603403708, 5)
Error in deepstate_fuzz_fun_analyze(fun_path, 1603403708, 5) : 
  could not find function "deepstate_fuzz_fun_analyze"
> seed_analyze <- RcppDeepState::deepstate_fuzz_fun_analyze(fun_path ,1603403708, 5)
running the executable .. 
cd /home/tdhock/R/RcppDeepState/inst/testpkgs/testSAN/inst/testfiles/rcpp_write_index_outofbound && valgrind --xml=yes --xml-file=/home/tdhock/R/RcppDeepState/inst/testpkgs/testSAN/inst/testfiles/rcpp_write_index_outofbound/5_1603403708/1603403708_log --tool=memcheck --leak-check=yes --track-origins=yes ./rcpp_write_index_outofbound_DeepState_TestHarness --seed=1603403708 --timeout=5 --fuzz > /home/tdhock/R/RcppDeepState/inst/testpkgs/testSAN/inst/testfiles/rcpp_write_index_outofbound/5_1603403708/seed_valgrind_log_text 2>&1

> seed_analyze
      inputs          logtable
1: <list[1]> <data.table[1x5]>
> seed_analyze$logtable[[1]]
              err.kind
1: Leak_DefinitelyLost
                                                                 message
1: 400 bytes in 1 blocks are definitely lost in loss record 125 of 1,334
                        file.line      address.msg          address.trace
1: write_index_outofbound.cpp : 7 No Address found No Address Trace found
tdhock commented 3 years ago 
(base) tdhock@maude-MacBookPro:~/R/RcppDeepState$ valgrind --version
valgrind-3.13.0
akhikolla commented 3 years ago

It is the issue with the seed value we provided. DeepState makes use of srand (standard C random number generator library) that's not guaranteed to be consistent across platforms.

The inputs generated by the seed 1603403708 are different in my laptop which caused an InvalidWrite(write_index_outofbound.cpp : 8) error whereas the inputs generated by 1603403708 are different in your platform which produced an Leak_DefinitelyLost(write_index_outofbound.cpp : 7).

The only way we can rectify this is by implementing our own pseudo-random number generators that are cross-platform consistent. There is an issue raised with the deepstate for this https://github.com/trailofbits/deepstate/issues/364.

Maybe I'll provide a note saying that the seed value is different across the platforms and the output may vary.