search

ossf / osv-schema

Open Source Vulnerability schema.
https://ossf.github.io/osv-schema/
Apache License 2.0
176 stars 75 forks source link

Add validation for CVSS #251

Closed andrewpollock closed 1 month ago

andrewpollock commented 2 months ago

Addresses undetected invalidity surfaced in https://github.com/github/advisory-database/blob/adf108ed87cfbe666a56cd9cab986afc3854150e/advisories/github-reviewed/2023/11/GHSA-jjfh-589g-3hjx/GHSA-jjfh-589g-3hjx.json and helps address https://github.com/google/osv.dev/issues/2369

h/t @gregsdennis for assistance with the validation syntax

This uses the regexes from the official schema definitions for CVSS, with some additional slash-escaping that regex101.com seemed to feel was necessary to make them valid.

$ git -C ~/gosst/osv/advisory-database/ checkout adf108ed87cfbe666a56cd9cab986afc3854150e
HEAD is now at adf108ed87c Publish GHSA-jjfh-589g-3hjx
$ ~/go/bin/jv ./validation/schema.json ~/gosst/osv/advisory-database/advisories/github-reviewed/2023/11/GHSA-jjfh-589g-3hjx/GHSA-jjfh-589g-3hjx.json 
schema ./validation/schema.json: ok

instance /usr/local/google/home/apollock/gosst/osv/advisory-database/advisories/github-reviewed/2023/11/GHSA-jjfh-589g-3hjx/GHSA-jjfh-589g-3hjx.json: failed
jsonschema validation failed with 'file:///usr/local/google/home/apollock/gosst/osv/osv-schema/validation/schema.json#'
- at '/severity/1': allOf failed
  - at '/severity/1/score': 'CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:N/I:N/A:L' does not match pattern '^CVSS:4[.]0/AV:[NALP]/AC:[LH]/AT:[NP]/PR:[NLH]/UI:[NPA]/VC:[HLN]/VI:[HLN]/VA:[HLN]/SC:[HLN]/SI:[HLN]/SA:[HLN](/'
$ git -C ~/gosst/osv/advisory-database/ checkout main
Previous HEAD position was adf108ed87c Publish GHSA-jjfh-589g-3hjx
Switched to branch 'main'
Your branch is up to date with 'origin/main'.
$ ~/go/bin/jv ./validation/schema.json ~/gosst/osv/advisory-database/advisories/github-reviewed/2023/11/GHSA-jjfh-589g-3hjx/GHSA-jjfh-589g-3hjx.json 
schema ./validation/schema.json: ok

instance /usr/local/google/home/apollock/gosst/osv/advisory-database/advisories/github-reviewed/2023/11/GHSA-jjfh-589g-3hjx/GHSA-jjfh-589g-3hjx.json: ok
andrewpollock commented 2 months ago

The fact that past me authored https://github.com/ossf/osv-schema/pull/178 (almost a year to the day) has somewhat floored me...

andrewpollock commented 2 months ago

👋 @frasertweedale since #168 is what begat #178 a ~year ago, I figured I'd run this PR over all of the Haskell security advisories, and HSEC-2023-0003 doesn't (to me, legitimately) validate:

instance /usr/local/google/home/apollock/gosst/osv/security-advisories/2023/HSEC-2023-0003.json: failed
jsonschema validation failed with 'file:///usr/local/google/home/apollock/gosst/osv/osv-schema/validation/schema.json#'
- at '/affected/0/severity/0': allOf failed
  - at '/affected/0/severity/0/score': 'CVSS:2.0/AV:N/AC:L/Au:N/C:P/I:P/A:P' does not match pattern '^((AV:[NAL]|AC:[LMH]|Au:[MSN]|[CIA]:[NPC]|E:(U|POC|F|H|ND)|RL:(OF|TF|W|U|ND)|RC:(UC|UR|C|ND)|CDP:(N|L|LM|MH|H'
frasertweedale commented 2 months ago

@andrewpollock thank you; that is indeed a bug. Will create a ticket our side and fix it soon!

frasertweedale commented 1 month ago

@andrewpollock this was resolved, via https://github.com/haskell/security-advisories/pull/218, and our published OSV data has been fixed. Can you verify on your end?

andrewpollock commented 1 month ago

@frasertweedale

Can you verify on your end?

Verified:

$ git -C ~/gosst/osv/osv-schema/ show --summary validation/schema.json
commit ce77bfbaba0315697e50dda222ca0a4de75cce33 (HEAD -> validate_cvss_severity, origin/validate_cvss_severity)
Author: Andrew Pollock <apollock@google.com>
Date:   Fri Jul 12 02:54:46 2024 +0000

    Correct the validation

    This now successfully fails to validate https://github.com/github/advisory-database/blob/adf108ed87cfbe666a56cd9cab986afc3854150e/advisories/github-reviewed/2023/11/GHSA-jjfh-589g-3hjx/GHSA-jjfh-589g-3hjx.json

    h/t @gregsdennis

    Signed-off-by: Andrew Pollock <apollock@google.com>

$ git -C security-advisories show --summary 2023/HSEC-2023-0003.json
commit 14197d1fabdb480e34f7424d68b96882210d084d (HEAD -> generated/osv-export, origin/generated/osv-export)
Author: Haskell Security Response Team <security-advisories@haskell.org>
Date:   Fri Jul 19 05:19:03 2024 +0000

    2024-07-19 05:19:03+00:00 (d09058a544bf45cc0814ed9b300cd940bc263617)

$ ~/go/bin/jv ~/gosst/osv/osv-schema/validation/schema.json security-advisories/2023/HSEC-2023-0003.json
schema /usr/local/google/home/apollock/gosst/osv/osv-schema/validation/schema.json: ok

instance security-advisories/2023/HSEC-2023-0003.json: ok