search

stg-tud / MUBench

Other
53 stars 30 forks source link

MuBench pipeline check dataset checks non passed datasets too #429

Open akwick opened 5 years ago

akwick commented 5 years ago

Done:

Observed output:

...
[WARNING] Unknown dataset entry "testng.677302c.grouminer-4" in dataset "FSE18-ExPrecision-TruePositives".
[WARNING] Unknown dataset entry "testng.677302c.grouminer-17" in dataset "FSE18-ExPrecision-TruePositives".
[WARNING] Unknown dataset entry "alibaba-druid.e10f28.1" in dataset "JCA-All".
[WARNING] Unknown dataset entry "alibaba-druid.e10f28.2" in dataset "JCA-All".
[WARNING] Unknown dataset entry "andoid-rcs-rcsjta.04d847.1" in dataset "JCA-All".
...

Expected output: No output for datasets which aren't passed.

akwick commented 5 years ago

I observed the same behavior for projects. Execute pipeline check dataset --only public-cms

Observed output:

mubench> pipeline check dataset --only public-cms
[INFO   ] Starting benchmark...
[INFO   ] All requirements satisfied. You're good to go.
[WARNING] Invalid dataset entry "closure.319" in "TSE17-ExPrecision". Entries must be "project.version.misuse".
[WARNING] Invalid dataset entry "itext.5091" in "TSE17-ExPrecision". Entries must be "project.version.misuse".
[WARNING] Invalid dataset entry "jmrtd.51" in "TSE17-ExPrecision". Entries must be "project.version.misuse".
....