Download all epss data, and import database. We can explore the data by SQL querys!
NOW: THIS IS AN EXPERIMENTAL IMPLEMENTATION.
README.md was created using Google Translate.
Supported data
EPSS is Exploit Prediction Scoreing Syste from FIRST ( https://www.first.org/epss/ ).
I want to analyze EPSS, but I don't need to use SIEM, so I wanted something that could be analyzed using SQL. We thought it was important to first implement something simple and have it widely used.
And The KEV catalog is now also included in the database. I think the range of use will be further expanded by combining it with EPSS's cveID.
An environment where Docker can be executed is required.
/opt/epss-db
|-- Documents
| |-- epss-graph.png
| `-- epss-graph_-a.png
|-- LICENSE
|-- README.md
|-- docker
| |-- Dockerfile
| |-- README.md
| `-- env
|-- epss-graph.sh
|-- init-script
| |-- epss-init.sh
| |-- kev-init.sh
| `-- vulnrichment-init.sh
|-- my.cnf
|-- queryConsole.sh
|-- skel
| `-- plot.plt
|-- subprogram
| |-- epss-add.sh
| `-- vulnrichUpdate.sh
|-- update-all.sh
|-- update-epss.sh
|-- update-kev.sh
`-- update-vulnrich.sh
Get Dockaer image
$ docker pull hogehuga/epss-db
Create docker volume
epssDB
volmeepssFile
volume$ docker volume create epssDB
$ docker volume create epssFile
Run container
-v <yourShredDirctory>:/opt/epss-db/share
.
-v /home/hogehuga/share:/opt/epss-db/share
$ docker container run --name epssdb -v epssDB:/var/lib/mysql -v epssFile:/opt/epss-db/epss-data -e MYSQL_ROOT_PASSWORD=mysql -d hogehuga/epss-db
Prepare the data
$ docker exec -it epssdb /bin/bash
(work inside a container)
# cd /opt/epss-db/init-script
# ./epss-init.sh
Once your data is ready, all you need to do is use it!
run EPSS container.
Init for The KEV Catalog database.
$ docker exec -it epssdb /bin/bash
(work inside a container)
# cd /opt/epss-db/init-script
# ./kev-init.sh
run EPSS container
Init for The Vulnrichment database
$ docker exec -it epssdb /bin/bash
(work inside a container)
# cd /opt/epss-db/init-script
# ./vulnrichment-init.sh
Enter the container and use SQL commands to perform analysis.
$ docker exec -it epssdb /bin/bash
(work inside a container)
# cd /opt/epss-db
# ./epssquery.sh
mysql> select * from epssdb limit 1;
+----+---------------+---------+------------+-------+------------+
| id | cve | epss | percentile | model | date |
+----+---------------+---------+------------+-------+------------+
| 1 | CVE-2020-5902 | 0.65117 | NULL | NULL | 2021-04-14 |
+----+---------------+---------+------------+-------+------------+
1 row in set (0.00 sec)
mysql>
Create EPSS and percentile charts and CSV data for the past 180 days.
-a
option will create the graph using all the data present in the database../skel/plot.plt
If you want to change gnuplot options, edit the skel-
# LANG=C.utf8 gnuplot skel-<CVE-ID>.plt
# ./epss-graph.sh -cve "CVE-2022-27016"
; -> ./share/CVE-2022-27016.csv (from:180 days ago)
; -> ./share/EPSS-CVE-2022-27016.png
# ./epss-graph.sh -cve "CVE-2022-27016" -a
; -> Similar to above, but creates images for all registered periods
Automatically registers data from the last registered data to the latest data in the database.
# ./epss-autoAdd.sh
git pull origin
or rebuild container.
# cd /opt/epss-db
# git pull origin
on HOST
$ docker stop epssdb
$ docker pull hogehuga/epss-db
$ docker container run --name epssdbNEWNAME -v epssDB:/var/lib/mysql -v epssFile:/opt/epss-db/epss-data -e MYSQL_ROOT_PASSWORD=mysql -d hogehuga/epss-db
; Please specify the same value as last time
NOTE:
- Databases(/var/lib/mysql as "epssDB" docker volume) and files(/opt/epss-db/epss-data as "epssFile" docker volume) will be inherited.
At the moment, we are using SQL.
$ docker exec -it epssdb /bin/bash
(work inside a container)
# cd /opt/epss-db
# ./epssquery.sh
mysql> select YEAR(dateAdded) as year, count(dateAdded) as count from kevcatalog group by year ;
+------+-------+
| year | count |
+------+-------+
| 2021 | 311 |
| 2022 | 555 |
| 2023 | 187 |
| 2024 | 51 |
+------+-------+
4 rows in set (0.00 sec)
mysql> select epssdb.cve, epssdb.epss, epssdb.percentile, kevcatalog.dateAdded, kevcatalog.vendorProject, kevcatalog.knownRansomwareCampaignUse from epssdb INNER JOIN kevcatalog ON epssdb.cve = kevcatalog.cveID where epssdb.cve="CVE-2021-44529" and epssdb.date="2024-04-20";
+----------------+---------+------------+------------+---------------+----------------------------+
| cve | epss | percentile | dateAdded | vendorProject | knownRansomwareCampaignUse |
+----------------+---------+------------+------------+---------------+----------------------------+
| CVE-2021-44529 | 0.97068 | 0.99757 | 2024-03-25 | Ivanti | Unknown |
+----------------+---------+------------+------------+---------------+----------------------------+
1 row in set (0.09 sec)
mysql>
Unlike CVSS etc., it does not provide differences, so please delete the database and re-register it.
# cd /opt/epss-db
# ./kev-refresh.sh
CVE-nnnn-nnnn
...
#
mysql> select adpSSVCAutomatable, count(*) from summary group by adpSSVCAutomatable;
+--------------------+----------+
| adpSSVCAutomatable | count(*) |
+--------------------+----------+
| no | 2653 |
| Yes | 558 |
| | 41 |
+--------------------+----------+
3 rows in set (0.01 sec)
mysql>
Since the data update status is unknown, please delete all data and register again.
# /opt/epss-db/update-vulnrich.sh
# /opt/epss-db/queryConsole.sh
> drop table richment;
# rm -rf /opt/epss-db/ulnrichment
Field | Type |
---|---|
id | int |
cve | varchar(20) |
epss | double |
percentile | double |
model | varchar(20) |
date | date |
https://www.cisa.gov/known-exploited-vulnerabilities-catalog
field | original json type | note | mysql Table |
---|---|---|---|
id | (not exist) | (for RDBMS) | int, not Null |
cveID | string | ^CVE-[0-9]{4}-[0-9]{4,19}$ | varchar(20) |
vendorProject | string | text | |
product | string | text | |
vulnerabilityName | string | text | |
dateAdded | string | format: YYYY-MM-DD | date |
shortDescription | string | text | |
requiredAction | string | text | |
dueDate | string | format: YYYY-MM-DD | date |
knownRansomwareCampaignUse | string | (Known or Unknown only?) | text |
notes | string | text |