Closed njshanahan closed 3 years ago
This is an expected behavior;
Understood. However, consider CWE-825. The stack component of the test was specifically added per our request (Issue #1044) to reflect the capabilities of the HARD solution. If this is a valid variant of the CWE, should the score not reflect a successful mitigation?
You're making a valid point. This is a very subjective and a non-empirical call. @austinhroach, please let us know if DARPA wants us to review all the multi-part tests and add a subjective weighted calculation to each CWE score.
@austinhroach Any thoughts regarding this?
These are both valid perspectives. Within DARPA, we've been discussing scoring for the purposes of reporting, and have come to the conclusion that it would be useful to have both a "percentage of tests passed" score for each CWE, in addition to the binary "was this CWE mitigated in all tests" score.
For the purposes of the "percentage of tests passed" score, given the above example we would say that for CWE-825 67% of the tests passed. If this was still the case when final results were reported, we would want to see something like "67% of the tests for CWE-825 were passed. This CWE was not completely mitigated, and here's the reason why."
Roger that; I have changed the ticket title to be more generic. I will keep the format as shown PartX: score(partX), ...)
, but the overall score of the CWE will be changed, which will be similar to what Nicholas has described above.
The HARD pipelines mitigated one or more parts of the following Resource Management tests, however, the reported score for each was
HIGH
vulnerability. Given the Buffer Error scoring range philosophy, shouldn't these tests have scoredLOW
orMED
?I would have expected the first five tests to have scored
MED
, because 50% of vulnerable test components were addressed (CWE-761 includes two benign parts). Similarly, I would have expected CWE-825 to have scoredLOW
because 67% of the test parts were mitigated. I appreciate your help.Tagging @austinhroach for awareness.