OWASP / OpenCRE

https://opencre.org
Creative Commons Zero v1.0 Universal
77 stars 33 forks source link

make `--upstream_sync` also sync the gap analysis graph progressively #534

Open northdpole opened 2 months ago

northdpole commented 2 months ago

Issue

What is the issue?

Current gap analysis graph is 9Gb which makes it very hard to download reliably when running locally (also you shouldn't need 9Gb of data to run the project)

Expected Behaviour

Upstream sync should either download the data necessary in order to run(graph structure). For the Gap Analysis graph it should either grab it from a place suitable for this sort of data (e.g. google public dataset) or it should load it from upstream opportunistically and then save it in the local database