google / osv.dev

Open source vulnerability DB and triage service.
https://osv.dev
Apache License 2.0
1.51k stars 188 forks source link

Visibility into update schedules from various data sources #1155

Open jayvdb opened 1 year ago

jayvdb commented 1 year ago

I couldnt find on https://osv.dev/ , or the repo README, where I look to easily see when the various data sources were last updated. I am expecting to see something like "crates.io: Latest sync competed 3 hrs ago" on the website, for each data source, if it varies for each data source, or "All data sources are synchronised daily" in the README. Something to provide users with a rough intuition on how/when updates are happening. I am guessing that there are live feeds of new advisories from some of these datasources - it would be good to know which ones, and whether the feed only contains new advisories or also contains updates to old advisories.

The reason I am looking is there has been a bug upstream in an old record https://github.com/github/advisory-database/issues/1800 , which has been updated upstream and I'd like to know whether it is now fixed in osv-scanner.

I can load https://osv-vulnerabilities.storage.googleapis.com/crates.io/GHSA-wcg3-cvx6-7396.json and look at its "Last-Modified" header. Oddly, it says it was last modified 4 hrs ago (which btw is older than the GHSA update, so I guess I need to wait some more), but when I look at the list at https://osv.dev/list?ecosystem=crates.io&q= , default ordered last modified descending, it doesn't appear there. I guess there are various definitions of "modified" in play here.

andrewpollock commented 1 year ago

@jayvdb yes, we will be working on addressing this deficiency as part of our SLO monitoring plan in Q2 of 2023, as we'll need a way to monitor this, and exposing it for use cases like you describe here makes total sense.

We've recently added our SLO to our FAQ.

Separately, there may be some issues with downstream data providers correctly updating the last_modified field of records when they change.

github-actions[bot] commented 3 months ago

This issue has not had any activity for 60 days and will be automatically closed in two weeks