search

guacsec / guac

GUAC aggregates software security metadata into a high fidelity graph database.
https://guac.sh
Apache License 2.0
1.29k stars 176 forks source link

[feature] Update deps.dev collector to take advantage of the new API #1768

Open pxp928 opened 8 months ago

pxp928 commented 8 months ago

Is your feature request related to a problem? Please describe.

With the new release of the API for deps.dev -> https://blog.deps.dev/api-v3/

https://github.com/guacsec/guac/blob/e2832062e88193e87c484710bcd512c48ceee3a2/pkg/handler/collector/deps_dev/deps_dev.go#L401-L419)

Describe the solution you'd like upgrade the current deps.dev collector to use the latest API features

pxp928 commented 6 months ago

@rakshitgondwal would you be interested in working on this?

rakshitgondwal commented 6 months ago

Sure @pxp928, I can take this up after #1710

pxp928 commented 5 months ago

@rakshitgondwal have you started any work on this? If not, @nathannaveen would like to take it over.

rakshitgondwal commented 5 months ago

Hi @pxp928, nope I didn't get the time to do so, and sure someone else can take this over.

nathannaveen commented 5 months ago

Hey @pxp928, could I work on this?

nathannaveen commented 4 months ago

Recently I have been working on this issue, and realize that since the deps.dev v3alpha version isn't stable it won't exactly fit our needs.

For example, when doing a purl batch lookup https://docs.deps.dev/api/v3alpha/#purllookupbatch, if any of the repos passed to the batch request aren't contained in the deps.dev database, it returns nil.

So the only way to check whether a purl is contained in the deps.dev database is to manually do a purlLookup for each individual purl. Which removes our need for the purlLookupBatch.

This issues will probably disappear when the deps.dev/api/v3alpha becomes stable. But for now, this won't work for what we want.

Additionally, we directly use the osv-scanner repo. https://github.com/guacsec/guac/blob/dad65eb57ce9ffe9db2d49ba3cda097a48191eda/pkg/certifier/osv/osv.go#L27

The issue with this, is that osv-scanner calls the deps.dev resolve API, which only works with deps.dev/api/v3 not v3alpha.

Note that I didn't realize we directly use the osv-scanner repo at the time of writing this issue https://github.com/guacsec/guac/issues/1947.

So I think the best way to go about this is to wait for the version to become stable and then implement it.

If you want to run some tests, this is the batch request that will return nil: https://github.com/guacsec/guac/compare/main...nathannaveen:guac:nathan/depsdevNewAPI#diff-82819d87163ba23dd5d158849f788068d5eadd6e1818517c879ec08b0745131cR455