Rscript throws error "missing value where TRUE/FALSE needed" when analyzing benchmarks

trivikr commented 4 years ago

Version: master
Platform: Ubuntu
Subsystem: http2

Followed documentation to create comparison benchmarks

Command run:

$ node benchmark/compare.js --old ../node-master --new ../node-http2-for-of --set benchmarker=wrk http2 > compare-http2.csv
[05:52:54|% 100| 5/5 files | 60/60 runs | 12/12 configs]: Done

Output file (rename extension to CSV): compare-http2.txt

I receive the following error when trying to analyze benchmarks:

$ cat compare-http2.csv | Rscript benchmark/compare.R
Error in if (w$p.value < 0.001) { : missing value where TRUE/FALSE needed
Calls: ddply ... llply -> loop_apply -> .Call -> <Anonymous> -> .fun
Execution halted

trivikr commented 4 years ago

The line where error is thrown https://github.com/nodejs/node/blob/8b4af64f50c5e41ce0155716f294c24ccdecad03/benchmark/compare.R#L81-L83

trivikr commented 4 years ago

This is probably related to using benchmarker=wrk in the command, and can be closed.

I would still prefer to keep it open to implement benchmark exit if data is not useful.

nodejs / node

Rscript throws error "missing value where TRUE/FALSE needed" when analyzing benchmarks #31045