The combine-to-osv job takes more than 1 hour to write files to the cloud, while debian-cve-convert (around 40k files) takes less than 1 minute. Although combine-to-osv processes more files than debian-cve-convert, the time difference is excessive. debian-cve-convert uses gsutil, and combine-to-osv uses gcloud SDK.
Attempt to replace gcloud with gsutil in combine-to-osv and conduct performance testing to determine if this improves performance.
The importer relies on blob.time_created to determine if GSC records need to be re-imported. Updating to gsutil should not affect the creation time; only the modification time will remain the same if there are no changes.
The
combine-to-osvjob takes more than 1 hour to write files to the cloud, whiledebian-cve-convert(around 40k files) takes less than 1 minute. Althoughcombine-to-osvprocesses more files thandebian-cve-convert, the time difference is excessive.debian-cve-convertusesgsutil, andcombine-to-osvuses gcloud SDK.Attempt to replace
gcloudwithgsutilincombine-to-osvand conduct performance testing to determine if this improves performance.The
importerrelies onblob.time_createdto determine if GSC records need to be re-imported. Updating togsutilshould not affect the creation time; only the modification time will remain the same if there are no changes.