Schema support for cve/description/domain uniqueness

SoftwareDesignLab / nvip-crawler

Crawler project for National Vulnerability Integlligence Platform

MIT License

0 stars 0 forks source link

Schema support for cve/description/domain uniqueness #185

Closed memeeerit closed 10 months ago

memeeerit commented 10 months ago

adds domain and description_hash columns to the rawdescription table.

The domain is determined by the URL library.

description_hash autopopulates upon insert with a trigger.

A uniqueness key was put on the tuple (cve_id, description_hash, domain) and no longer needs to be checked individually for each CVE before insertion.

memeeerit commented 10 months ago

This PR also includes an edit to a past liquibase changeset which violates its hash. To deploy this that changeset hash needs to be cleared in the database.