search

vulsio / go-cve-dictionary

Build a local copy of CVE (NVD and Japanese JVN). Server mode for easy querying.
Apache License 2.0
368 stars 109 forks source link

fix(fetch): fix first CVE year #380

Closed jdorel closed 7 months ago

jdorel commented 7 months ago

What did you implement:

The first CVE was created in 1999, the previous feed fetch only contained CVE from 2002 onward. But with the API we get the CVE from 1999, 2000 and 2001

Type of change

How Has This Been Tested?

before

$ go-cve-dictionary fetch nvd
INFO[04-24|19:46:22] Inserting NVD into DB (sqlite3). 
INFO[04-24|19:46:22] Deleting NVD tables... 
INFO[04-24|19:46:22] Fetching CVE information from NVD. 
INFO[04-24|19:46:22] Fetching... https://github.com/vulsio/vuls-data-raw-nvd-api-cve/archive/refs/heads/main.tar.gz 
INFO[04-24|19:46:41] Fetching CVE information from NVD(2002). 
INFO[04-24|19:46:42] Inserting fetched CVEs(2002)... 
2392 / 2392 [------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------] 100.00% 1270 p/s
INFO[04-24|19:46:44] Refreshed 2392 CVEs. 
INFO[04-24|19:46:44] Fetching CVE information from NVD(2003). 
INFO[04-24|19:46:44] Inserting fetched CVEs(2003)... 
1553 / 1553 [------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------] 100.00% 1314 p/s
INFO[04-24|19:46:46] Refreshed 1553 CVEs. 
INFO[04-24|19:46:46] Fetching CVE information from NVD(2004). 
INFO[04-24|19:46:46] Inserting fetched CVEs(2004)... 
2707 / 2707 [-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------] 100.00% 824 p/s
...

after

$ go-cve-dictionary fetch nvd
INFO[04-24|19:47:26] Inserting NVD into DB (sqlite3). 
INFO[04-24|19:47:26] Deleting NVD tables... 
INFO[04-24|19:47:26] Fetching CVE information from NVD. 
INFO[04-24|19:47:28] Fetching... https://github.com/vulsio/vuls-data-raw-nvd-api-cve/archive/refs/heads/main.tar.gz 
INFO[04-24|19:47:56] Fetching CVE information from NVD(1999). 
INFO[04-24|19:47:57] Inserting fetched CVEs(1999)... 
1579 / 1579 [------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------] 100.00% 2385 p/s
INFO[04-24|19:47:57] Refreshed 1579 CVEs. 
INFO[04-24|19:47:57] Fetching CVE information from NVD(2000). 
INFO[04-24|19:47:58] Inserting fetched CVEs(2000)... 
1242 / 1242 [------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------] 100.00% 2487 p/s
INFO[04-24|19:47:59] Refreshed 1242 CVEs. 
INFO[04-24|19:47:59] Fetching CVE information from NVD(2001). 
INFO[04-24|19:47:59] Inserting fetched CVEs(2001)... 
1556 / 1556 [------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------] 100.00% 1571 p/s
INFO[04-24|19:48:00] Refreshed 1556 CVEs. 
INFO[04-24|19:48:00] Fetching CVE information from NVD(2002). 
INFO[04-24|19:48:01] Inserting fetched CVEs(2002)... 
2392 / 2392 [------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------] 100.00% 1609 p/s
INFO[04-24|19:48:02] Refreshed 2392 CVEs. 
INFO[04-24|19:48:02] Fetching CVE information from NVD(2003). 
INFO[04-24|19:48:03] Inserting fetched CVEs(2003)... 
1553 / 1553 [------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------] 100.00% 1656 p/s
INFO[04-24|19:48:04] Refreshed 1553 CVEs. 
INFO[04-24|19:48:04] Fetching CVE information from NVD(2004). 
INFO[04-24|19:48:04] Inserting fetched CVEs(2004)... 
2707 / 2707 [------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------] 100.00% 1375 p/s
...

Checklist:

You don't have to satisfy all of the following.

Is this ready for review?: YES

Reference

MaineK00n commented 7 months ago

This is probably because go-cve-dictionary used to use JSON Feed(https://nvd.nist.gov/vuln/data-feeds#JSON_FEED), starting in 2002, but since vulsio/vuls-data-raw-nvd-api-cve(https://github.com/vulsio/vuls-data-raw-nvd-api-cve), they forgot to change it, even though the CVE year is from 1999.