Motivation behind new security advisory schema standard in this repo

[mprpic]

Hi,

I recently happened to come across the vulnerability JSON schema in this repo that is supposed to standardize exchange of security advisories in machine-readable format. Is there any reason to duplicate this effort and not promote the use of an existing standard?

The CVRF spec (currently v1.3) that exists today was designed exactly for this purpose:

https://docs.oasis-open.org/csaf/csaf-cvrf/v1.2/csaf-cvrf-v1.2.html

and a lot of work is being put into its next iteration, CSAF 2.0 (currently in draft form):

https://github.com/oasis-tcs/csaf/blob/master/csaf_2.0/json_schema/csaf_json_schema.json

The CSAF schema is both more comprehensive and could be adopted by all current users of CVRF.

Are there any specific problems that the schema in this repo is trying to solve that CVRF/CSAF are not? Are there any improvements that could be made to CSAF 2.0 to meet the goals of this work group?

Thanks!

[Foxboron]

I'm a bit curious how the oasis spec overlaps with the current efforts of the MITRE automation WG. https://github.com/CVEProject/automation-working-group/tree/master/cve_json_schema/v5.x_discuss

Is there any collaboration, or consensus, if these are competing standards or if they compliment each other?

[joshbressers]

I imagine all of these are this https://xkcd.com/927/

I didn't know CSAF 2.0 even existed until just now. It's hard to keep track of all these things. I mean, maybe just keeping track of things could be a goal of this group.

That said, the the CSAF schema looks VERY heavy, possibly too heavy to anyone other than an enterprise to adopt and use. A lot of these sort of standards are too difficult for most open source projects to adopt.

Whatever this group does, it's imperative to make "zero friction" part of the DNA.

[mprpic]

I'm a bit curious how the oasis spec overlaps with the current efforts of the MITRE automation WG. https://github.com/CVEProject/automation-working-group/tree/master/cve_json_schema/v5.x_discuss

Is there any collaboration, or consensus, if these are competing standards or if they compliment each other?

@Foxboron That schema has a different scope: it's supposed to represent a single vulnerability in machine-readable format. The CSAF schema represents a security advisory, which in loose terms is a "collection" of vulnerabilities and their relationships to products that the advisory describes. There is some overlap in the way vulnerabilities are represented in the CSAF schema and in the CVE schema; for example, both have a list of acknowledgements. And I suppose a case could be made that the vulnerabilities array in the CSAF schema could just point to the CVE schema, much like it does for CVSS scores. Although, I imagine that the vulnerability information in a CSAF document may represent be a subset of all the information present for a particular CVE, for which an explicit vulnerability entry may be useful.

@joshbressers

I imagine all of these are this https://xkcd.com/927/

I didn't know CSAF 2.0 even existed until just now. It's hard to keep track of all these things. I mean, maybe just keeping track of things could be a goal of this group.

That said, the the CSAF schema looks VERY heavy, possibly too heavy to anyone other than an enterprise to adopt and use. A lot of these sort of standards are too difficult for most open source projects to adopt.

Whatever this group does, it's imperative to make "zero friction" part of the DNA.

It being "heavy" shouldn't be the sole argument for creating an entirely new schema. It's an open standard so if you feel it's too complicated, suggest changes how to make it less so (and right now is the time to do it since 2.0 is still in draft!). It's designed the way it is to cover a lot of use cases and provide plenty of information. If it's difficult to adopt, then let's write tooling that makes it easier to do so!

Also, if you're looking at the example in the CSAF repo, it can be overwhelming since it represents an OpenShift release that consists of a ton of components. For comparison, here is a quick hypothetical security advisory for the latest Django security release with the bare minimum of information (covering only one of the fixed CVEs for brevity):

{
  "document": {
    "csaf_version": "2.0",
    "title": "Django security releases: 3.1.1, 3.0.10, 2.2.16",
    "publisher": "Django Security Team",
    "type": "Security Advisory",
    "tracking": {
      "id": "DJANGO-SEC-2020:1234",
      "status": "final",
      "version": "1",
      "initial_release_date": "2020-01-01T00:00:00Z",
      "current_release_date": "2020-01-01T00:00:00Z",
      "revision_history": [
        {
          "number": "1",
          "date": "2020-01-01T00:00:00Z",
          "description": "Current version"
        }
      ]
    },
    "references": [
      {
        "url": "https://www.djangoproject.com/weblog/2020/sep/01/security-releases/",
        "description": "Security advisory web URL."
      }
    ]
  },
  "product_tree": {
    "full_product_names": [
      {
        "product_id": "DJANGO-PID-1",
        "name": "Django 3.1.1"
      },
      {
        "product_id": "DJANGO-PID-2",
        "name": "Django 3.0.10"
      },
      {
        "product_id": "DJANGO-PID-3",
        "name": "Django 2.2.16"
      }
    ]
  },
  "vulnerabilities": [
    {
      "cve": "CVE-2020-24583",
      "product_status": {
        "fixed": [
          "DJANGO-PID-1",
          "DJANGO-PID-2",
          "DJANGO-PID-3"
        ]
      }
    }
  ]
}

90% of this is boilerplate that can be generated by a tool whose input could literally be a list of CVEs, the versions in which they are fixed, and w/e title you want to provide for the advisory. Obviously, providing more information than the bare minimum should be encouraged.

[Foxboron]

Is there any adoption of the oasis schema currently? The RedHat advisory JSON API seems to be loosely(?) based on CVRF but I take it 1.2 was never adopted as JSON?

Has there been any effort check if NVD would standardize the current JSON dumps to use the OASIS schema?

[mprpic]

Is there any adoption of the oasis schema currently? The RedHat advisory JSON API seems to be loosely(?) based on CVRF but I take it 1.2 was never adopted as JSON?

Red Hat publishes CVRF documents using the 1.2 schema, which are available here:

https://www.redhat.com/security/data/cvrf/

The Security Data API serves those same files in XML but also transform it to JSON, e.g.:

https://access.redhat.com/labs/securitydataapi/cvrf/RHSA-2019:3200.json

That JSON doesn't use the CSAF 2.0 spec though, it's merely a 1:1 copy of the XML ran through a xml-to-json conversion.

Once the 2.0 spec is finalized, I'm sure we'll look at deprecating the old XML files and replacing them with the JSON-only ones both on the data/cvrf/ file share and in the API.

Has there been any effort check if NVD would standardize the current JSON dumps to use the OASIS schema?

NVD is a database of vulnerabilities, not advisories. So there is nothing for them to adopt from the CSAF schema. They do publish individual CVEs using the CVE JSON schema mentioned earlier:

https://nvd.nist.gov/vuln/data-feeds#JSON_FEED

[JasonKeirstead]

Just to be clear @Foxboron , don't get confused by the new name - CSAF 2.0 is the successor to CVRF 1.2. It was made by the same stakeholder group. Don't view them as competing, it is the successor.

[Foxboron]

@mprpic

Right, so CVRF only solves one part of the problem which is advisory consumption. But the main issue (in my opinion) is CVE consumption. So even if we did standardize on CVRF for advisories, consuming CVE data would still be left to the will of whoever publishes the data. This means the CVE JSON object between RedHat, NVD, Mitre or Vendor XYZ is still going to be in different formats. It seems like redhat publishes a partial object for the CVE data, but I haven't looked too closely on it.

Is there any intention of standardize on a format for CVE so it's the same across the ecosystems? I feel like that was the original goal of the MITRE schema and the schema worked on by this WG.

@JasonKeirstead I didn't see they changed the name, so I thought it was "CVRF 2.0" :smile: But yes, noted! Thanks!

[joshbressers]

I've been thinking about this a bit, and this may be a topic for the purpose of this group.

An advisory contains vulnerabilities. A vulnerability might have a CVE or might not. It is a different discussion as to if all vulnerabilities should have a CVE, today they do not and that isn't getting fixed quickly.

Is the group's focus vulnerabilities or advisories? They're similar, but I think the difference is important.

[mprpic]

@Foxboron Red Hat's JSON representation for a CVE (example) predates the CVE schema from MITRE and closely matches the data we want displayed on our CVE pages (example). It also includes some data that is outside of scope of a general CVE representation.

With that being said, I don't see why we couldn't migrate to use the MITRE CVE schema. We'll have to do that gradually as we probably have several users of the current API, but the ultimate goal should be to publish the information in an industry-accepted standard rather than roll our own. We already publish CVRF and OVAL metadata, which are industry standards, so we might as well do the same for CVEs. The custom bits that we include in our API right now and don't fit in the MITRE CVE schema could use the newly-proposed x_ prefix attributes (https://github.com/CVEProject/automation-working-group/pull/92) to allow some flexibility.

[Foxboron]

@mprpic Let me clarify that I'm not explicitly talking about redhat, I'm just motioning towards them as they seem to have implemented parts of the CVRF 1.3 spec. I'm more thinking about OASIS standardizing on a subset of the vulnerability schema for CVE representation as part of CSAF 2.0, which then can be utilized by MITRE, NVDE, RedHat and so on. I'm a bit weary if having multiple specs for advisories and CVEs is a good idea or not.

I realize you do work for oasis and Redhat so it's probably a bit hard to be explicit about which part I'm referring to :)

[chandanbn]

CVE-JSON schema is an atomic representation of vulnerability information. V5 supports multiple perspectives on the same vulnerability from different vendors and researchers to coexist.

CSAF is a molecular representation of one or more vulnerabilities, mainly from one vendor's perspective.

I do not see them as competing standards.

To me, the schema here looks like a fork of the CVE-JSON with few additional fields covering OSS use cases. I like the impacted_methods and impacted_apis fields. CVE JSON does not capture that resolution.

[NicoleSchwartz]

After discussion in today's working group I am inclined to think we should mostly be finding existing items and pointing people to them (at first) and perhaps in future helping add tooling to them (if needed), but i don't feel that we should be designing anything novel.

[chandanbn]

If you need to capture impacted methods/APIs related to a vulnerability, please suggest enhancing CVE JSON format (via https://github.com/CVEProject/automation-working-group/). Descriptions of many CVEs (often in the libraries) do often cite the vulnerable method/call/function - it would be beneficial to capture those in machine readable format where available.

[MarcinHoppe]

Per agreement in the last WG meeting we decided to focus on existing standards.

@mprpic are you OK to close this issue with the ☝️ conclusion? Then I would open a new issue to catalog existing standards.

[mprpic]

@MarcinHoppe Absolutely! I would also simply remove the vulnerability.schema.json file from this repo. I don't see it documented anywhere nor used by any existing tooling so I don't think much harm will be done if we remove it.

I'd be happy to contribute to an existing catalog of vulnerability metadata formats.

[MarcinHoppe]

@mprpic I just opened #67 to start this work.