search

opencve / opencve

CVE Alerting Platform
https://www.opencve.io
Other
1.81k stars 227 forks source link

Include RESERVED CVEs #141

Open gitmopp opened 3 years ago

gitmopp commented 3 years ago

Enhacement OpenCVE uses https://nvd.nist.gov/feeds/json/cve/1.1/nvdcve-1.1-modified.json.gz to get the list of CVEs. The list does not include CVEs with the status RESERVED

Why it is required? The file https://cve.mitre.org/data/downloads/allitems.csv has over 46000 RESERVED CVEs. All of them are missing in OpenCVE. Even critical patches like https://msrc.microsoft.com/update-guide/vulnerability/CVE-2021-40444 are RESERVED, so you might miss critical notification.

Do you have a solution in mind, or a suggestion to improve OpenCVE? The lists on https://cve.mitre.org/data/downloads/index.html include RESERVED CVEs. It would be nice to use this list as an additional source. Unfortunately it seems not to have a meta file to detect file changes. The second problem is the RESERVED CVEs do not have a rating.

ncrocfer commented 3 years ago

Hi @gitmopp ,

Thank you for this very interesting feedback !

Just before starting I have a question (you seem to know the CVE ecosystem, I take this opportunity to ask you it ^^): OpenCVE.io currently handles 170518 CVE (see https://www.opencve.io/cve), and the MITRE displays TOTAL CVE Records: 160713. Do I miss something?

For the explanations: OpenCVE executes a 1st import using the NVD Data feeds: https://github.com/opencve/opencve/blob/f7a9c4df50d039541c1aabd8f20e74070febbd07/opencve/commands/imports/cve.py#L17

Then the nvdcve-1.1-modified.json.gz file is used to periodically import new CVEs and update existing ones. How OpenCVE.io can handle more CVEs than the MITRE ? There is no duplicates:

opencve=# SELECT count(distinct(cve_id)) FROM cves;
 count
--------
 170518
(1 row

Concerning your initial issue: indeed OpenCVE only relies for now on the NVD database, and as you said this list doesn't include the reserved CVEs.

I could periodically fetch the allitems.csv file provided by the MITRE, but first it's a really big file, and then this will only add not really useful information like:

CVE-2021-40444,Candidate,"** RESERVED ** This candidate has been reserved by an organization or individual that will use it when announcing a new security problem. When the candidate has been publicized, the details for this candidate will be provided.","",Assigned (20210902),"None (candidate not yet proposed)",""

I'm not sure of the benefit for the users. For example here the Microsoft detailled information is not displayed, no added-value to have the Reserved CVE displayed in the interface.

But I have a good news :) I plan to add a source feature: Debian DSA, ExploitDB, Microsoft Advisories, etc.

For example we can parse the https://msrc.microsoft.com/update-guide/vulnerability page for Microsoft and add new vulnerabilities. Each vulnerability is linked to 1 or more CVE, and if the CVE is not already existing OpenCVE workers will create it.

What do you think of this feature ? For you is-it something that could be useful or do you see other idea/process method to fetch all the vulnerabilities information ?

gitmopp commented 3 years ago

Hi,

https://cve.mitre.org/data/downloads/ reports 160713 CVE records. I have no idea how get 160713 records. Some PowerShell commands to parse the CSV file:

PS> Invoke-WebRequest -Uri https://cve.mitre.org/data/downloads/allitems.csv -OutFile allitems.csv
PS> $cve=Get-Content -Path .\allitems.csv | Select-Object -Skip 10 | ConvertFrom-Csv -Delimiter ',' -Header "Name","Status","Description","References","Phase","Votes","Comments"
PS> $cve.Count
216851
PS> ($cve.Status | where { $_ -eq "Entry" }).Count
3053
PS> ($cve.Status | where { $_ -eq "Candidate" }).Count
213798
PS> ($cve.Description | where { $_ -match "^\*\* RESERVED \*\*" }).Count
46190
PS> ($cve.Description | where { $_ -match "^\*\* DISPUTED \*\*" }).Count
879
PS> ($cve.Description | where { $_ -match "^\*\* REJECT \*\*" }).Count
9947

I like the idea of parsing other sources like https://msrc.microsoft.com/update-guide/vulnerability, but of course more interfaces will increase the maintenance effort.

ncrocfer commented 3 years ago

If someone else has an idea why such a diff between OpenCVE database and MITRE I will be happy to discuss that :)

gitmopp commented 3 years ago

I compared the CVEs between cve.mitre.org and OpenCVE and some CVEs are missing. Of course all RESERVED ones, but also these CVEs (the list might not be complete).

The CVEs are missing in https://nvd.nist.gov/feeds/json/cve/1.1/nvdcve-1.1-modified.json.gz . But they are registered at MITRE e.g., https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-35675

This is nothing OpenCVE can solve.

pierreyvesv commented 2 years ago

That's the same for "Dirty Pipe" CVE CVE-2022-0847 which exists on cve.mitre.org but is absent from nvd.nist.gov json.

ncrocfer commented 2 years ago

Yes, for me the best solution in this case is to support other sources, like the RedHat portal (CVE-2022-0847) or the Debian Security tracker (CVE-2022-0847).

Because the NVD is of course not enough, but for me the MITRE is not really helpful when the CVE is still marked as RESERVED.