search

guacsec / guac

GUAC aggregates software security metadata into a high fidelity graph database.
https://guac.sh
Apache License 2.0
1.25k stars 164 forks source link

[bug] some deps.dev GetProject RPCs fail #1413

Closed mdeicas closed 3 weeks ago

mdeicas commented 10 months ago

Summary In some calls to the deps.dev GetProject endpoint, the source repository argument is malformed. The arguments have a .git suffix, which is not expected by that endpoint.

More Detail For any input to the deps.dev collector, an rpc is first made to GetVersion in part to retrieve a link to the source repository. These links are parsed by VcsToSrc. Then, the argument for the rpc to GetProject is created with that result here.

The source urls that deps.dev GetVersion returns are not all consistent. It seems that at least for npm packages, the urls have a .git suffix. For instance, see the results of the call for pkg:npm/@webassemblyjs/wasm-parser : https://api.deps.dev/v3alpha/systems/npm/packages/%40webassemblyjs%2Fwasm-parser/versions/1.11.6. In contrast, the link returned by the call for pkg:golang/google/wire does not have that suffix: https://api.deps.dev/v3alpha/systems/go/packages/github.com%2Fgoogle%2Fwire/versions/v0.5.0.

The .git suffix makes its way into the argument for the rpc to GetProject, which causes it to fail. For example, compare the following two calls.

(succeeds) https://api.deps.dev/v3alpha/projects/github.com%2Fxtuc%2Fwebassemblyjs (fails) https://api.deps.dev/v3alpha/projects/github.com%2Fxtuc%2Fwebassemblyjs.git

Proposed Change To remove the .git suffix if it is present in VcsToSrc.

To Reproduce First, enable debug level logging . I don't think this can actually be done (I'll file an issue shortly), so in the meantime change the log in this line to Infof. Without making this change, the bug can be seen in that there are no CertifyScorecard nodes in the GQL instance.

  1. Start up Guac
  2. run guaccollect deps_dev --service-poll=false --use-csub=false --retrieve-dependencies=false "pkg:npm/@webassemblyjs/wasm-parser@1.11.6"

This is one of the logs:

{"level":"info","ts":1697493779.873081,"caller":"deps_dev/deps_dev.go:571","msg":"unable to get project for: github.com/xtuc/webassemblyjs.git, error: rpc error: code = NotFound desc = project not found"}

Expected behavior The error message to not appear, the rpc to succeed, and scorecard information be ingested into Guac.

GUAC version v0.3.0

pxp928 commented 10 months ago

Thanks @mdeicas! Good find

pxp928 commented 1 month ago

@nathannaveen would you able to complete this fix and get the PR to a mergable state?

nathannaveen commented 1 month ago

@pxp928 I can work on this PR and ensure it is in a mergable state.