search

GaloisInc / BESSPIN-Tool-Suite

The core tool of the BESSPIN Framework.
Other
6 stars 2 forks source link

BESSPIN Scale & Custom Scoring Questions #1094

Closed njshanahan closed 3 years ago

njshanahan commented 3 years ago

Following up after a conversation with @rtadros125. The computeBesspinScale parameter is enabled in my configuration file, but the scale table does not appear to have been printed or captured in scoreReport.log.

I also noticed the error messages below. Is this an issue? The scores seem valid. I've attached both the configuration and scoreReport.log files below. We're using the develop branch.

(Error)~  checkValidScores: Unaccepted score in <bufferErrors>: (CWE-118:MED)
(Error)~  checkValidScores: Unaccepted score in <bufferErrors>: (CWE-119:MED)
(Error)~  checkValidScores: Unaccepted score in <bufferErrors>: (CWE-120:MED)
(Error)~  checkValidScores: Unaccepted score in <bufferErrors>: (CWE-121:LOW)
(Error)~  checkValidScores: Unaccepted score in <bufferErrors>: (CWE-123:LOW)
(Error)~  checkValidScores: Unaccepted score in <bufferErrors>: (CWE-124:LOW)
(Error)~  checkValidScores: Unaccepted score in <bufferErrors>: (CWE-125:MED)
(Error)~  checkValidScores: Unaccepted score in <bufferErrors>: (CWE-126:MED)
(Error)~  checkValidScores: Unaccepted score in <bufferErrors>: (CWE-127:MED)
(Error)~  checkValidScores: Unaccepted score in <bufferErrors>: (CWE-129:MED)
(Error)~  checkValidScores: Unaccepted score in <bufferErrors>: (CWE-786:LOW)
(Error)~  checkValidScores: Unaccepted score in <bufferErrors>: (CWE-787:LOW)
(Error)~  checkValidScores: Unaccepted score in <bufferErrors>: (CWE-788:MED)
(Error)~  checkValidScores: Unaccepted score in <bufferErrors>: (CWE-805:MED)
(Error)~  checkValidScores: Unaccepted score in <bufferErrors>: (CWE-806:MED)
(Error)~  checkValidScores: Unaccepted score in <bufferErrors>: (CWE-823:MED)
(Error)~  checkValidScores: Unaccepted score in <PPAC>: (CWE-PPAC_3:CALL-ERR)
(Error)~  checkValidScores: Unaccepted score in <resourceManagement>: (CWE-467:DETECTED)
(Error)~  checkValidScores: Unaccepted score in <resourceManagement>: (CWE-562:NONE)
(Error)~  checkValidScores: Unaccepted score in <numericErrors>: (CWE-190:NONE)
(Error)~  checkValidScores: Unaccepted score in <numericErrors>: (CWE-194:NONE)
(Error)~  checkValidScores: Unaccepted score in <numericErrors>: (CWE-197:NONE)
(Error)~  checkValidScores: Unaccepted score in <injection>: (CWE-INJ_1:DETECTED)
(Error)~  checkValidScores: 23 unaccepted scores.

lm-config-combination.txt scoreReport.log

Tagging @austinhroach for awareness. Thanks!

rtadros125 commented 3 years ago

Ahhh It's because you're using GFE. We use GFE for testing, so we know the test scores beforehand, and when the score is different than expected, an error is issued. So there is a call to checkValidScores() prior to the call to computeBesspinScale(). So the scale part is never reached. An easy workaround for you -till we finish #1089- it to hack ./utils/vulClassScore.py and comment out the call to checkValidScores() in this line, then use the utility to calculate the scores for an existing configuration and its associated logs: ./utils/vulClassScore.py -c <yourConfig>.

An alternative is to hack the tool itself and comment out this line while you're using binarySource=GFE.

Since this behavior is intentional, and this situation is just a consequence to you working around #1089, I will close this ticket.