Crawler source errors - Githubissues

SoftwareDesignLab / nvip-crawler

Crawler project for National Vulnerability Integlligence Platform

MIT License

0 stars 0 forks source link

Crawler source errors #81

Closed atsawtelle closed 1 year ago

atsawtelle commented 1 year ago

Updated the whitelist
Cleaned up nvip-seeds.txt
Added user agent to the crawler to fix some 403s