OWASP-Benchmark / BenchmarkUtils

OWASP Benchmark Project Utilities - Provides scorecard generation and crawling tools for Benchmark style test suites.
https://owasp.org/www-project-benchmark/
GNU General Public License v3.0
13 stars 48 forks source link

Improve parser for wapiti json #72

Closed Qwarctick closed 2 weeks ago

Qwarctick commented 1 month ago

Hello, I'm working with Wapiti and I'm having some trouble generating scorecards in json format correctly.

I saw that you had a parser for Wapiti's json format, but it doesn't seem to have been updated since a long time (https://github.com/OWASP-Benchmark/BenchmarkUtils/blob/main/plugin/src/main/java/org/owasp/benchmarkutils/score/parsers/WapitiJsonReader.java). The json example in the tests returns 0% for all categories, so I'm pretty confident that the parser doesn't work very well.

I can work on it but I can't find any information about the corresponding tags. For example, CRLF Injection is for label 93 but what does that mean?

Thanks in advance :)

davewichers commented 1 month ago

@darkspirit510 - can you help @Qwarctick with this? @Qwarctick - can you email a copy of your Wapiti JSON results file (along with the script/instructions on how you generated it), to: dave.wichers@owasp.org and I'll forward to @darkspirit510 and he'll likely just fix it for you. He's generally super quick on stuff like this.

darkspirit510 commented 1 month ago

Hi @Qwarctick,

thank you for reaching out. Maybe you misunderstood the test files. Did you use the one from the results dir for the tests at https://github.com/OWASP-Benchmark/BenchmarkUtils/tree/main/plugin/src/test/resources/testfiles? Those intentionally do not contain ANY real finding to honor commercial tools disclousure policies. So to just prevent any issues here, the test files are just fake and contain one or two dummy results to test against. If you want to know how well a tool scores, you'll have to create a result file on your own.

Yes, the BenchmarkJava project contains some (older) actual result files, but those are all from open source tools and to test the whole BenchmarkJava and BenchmarkUtils project.

Qwarctick commented 1 month ago

Ok, thanks for the information. I can see why it's done that way.

I'm generating a json file. I'll send it to you at address dave.wichers@owasp.org as soon as it's ready.

darkspirit510 commented 3 weeks ago

@Qwarctick My PR got merged, please recheck your result file and see if this issue can be closed 😊

Qwarctick commented 2 weeks ago

Maybe I failed somewhere, but I tested with the json file I sended you by email.

Without the PR: Benchmark_v1 2_Scorecard_for_Wapiti_v608717d4a9f64e88bdc6faa796d37856

With the PR: Benchmark_v1 2_Scorecard_for_Wapiti_vmerged

I built the sources with mvn install, then used the script createScorecards.sh

darkspirit510 commented 2 weeks ago

Are you sure that you pulled BenchmarkUtils correctly? I just did a clean checkout, mvn install and generated the scorecard. I get results, as expected even better than yours.

Benchmark_v1 2_Scorecard_for_Wapiti_vmerged

Qwarctick commented 2 weeks ago

My bad, I used a wrong expectedresults file. It works well. Well done !