search

google / osv.dev

Open source vulnerability DB and triage service.
https://osv.dev
Apache License 2.0
1.49k stars 186 forks source link

Vulns from REST sources may be missed due to timing #2670

Open michaelkedar opened 5 days ago

michaelkedar commented 5 days ago

The importer uses last_update_date to compare both the Last-Modified of the HEAD request and the modified time of the individual records. last_update_date gets set to the time when the importer started processing the source.

This can cause an edge case where we miss a few updates:

00:10 - REST data source updates.
00:15 - Importer runs. Sets last_update_date to 00:15
00:20 - REST data source updates. Adds records with modified dates between 00:10 - 00:20.
00:30 - Importer runs. Processes the vulnerabilities with modified dates > 00:15 (last_update_date).

Here, the importer won't detect the vulnerabilities from the REST source that were modified between 00:10 - 00:15.

andrewpollock commented 5 days ago

last_update_date gets set to the time when the importer started processing the source

That's at the end of the run, not the start. Does that change your hypothesis at all?

michaelkedar commented 5 days ago

last_update_date gets set to the time when the importer started processing the source

That's at the end of the run, not the start. Does that change your hypothesis at all?

The value is set at the start, but either way, no - there'd be an issue as long as last_import_date is set anywhere between the two REST updates.