ossf / scorecard

OpenSSF Scorecard - Security health metrics for Open Source
https://scorecard.dev
Apache License 2.0
4.58k stars 497 forks source link

BUG: CI-Tests: internal error: internal error: Client.Repositories.ListStatuses: error getting commit statuses #3701

Open adam-moss opened 11 months ago

adam-moss commented 11 months ago

Describe the bug A clear and concise description of what the bug is.

Error: check runtime error: CI-Tests: internal error: internal error: Client.Repositories.ListStatuses: error getting commit statuses: GET https://gitlab.com/api/v4/projects/<redact>/repository/commits/statuses: 404 {message: 404 Commit Not Found}
2023/11/28 21:12:32 error during command execution: check runtime error: CI-Tests: internal error: internal error: Client.Repositories.ListStatuses: error getting commit statuses: GET https://gitlab.com/api/v4/projects/<redact>/repository/commits/statuses: 404 {message: 404 Commit Not Found}

Reproduction steps Steps to reproduce the behavior:

  1. Create a private repository on gitlab.com
  2. Run scorecard --repo gitlab.com/path/to/repo

Expected behavior A clear and concise description of what you expected to happen.

scorecard completes successfully with no errors reported.

Additional context I have built scorecard from source with the latest commit on the main branch.

naveensrinivasan commented 10 months ago

@adam-moss I tried to recreate and I wasn't able to.

Can you please add some specifics as to what of repository? Did it have any code? Did it have any commits?

Thanks

spencerschrock commented 10 months ago

@adam-moss I tried to recreate and I wasn't able to.

This may be relevant https://github.com/ossf/scorecard/issues/3581#issuecomment-1769369405

aarongoldenthal commented 7 months ago

I hit this today with Scorecard 4.13.1 at https://gitlab.com/gitlab-ci-utils/gitlab-pmd-cpd (and most of the others I tried in the same group, which are all public).

The error was under CI-Tests:

internal error: internal error: Client.Repositories.ListStatuses: https://github.com/ossf/scorecard/blob/49c0eed3a423f00c872b5c3c9f1bbca9e8aae799/docs/checks.md#ci-tests
error getting commit statuses: GET
https://gitlab.com/api/v4/projects/22151381/repository/commits/statuses
404 {message: 404 Commit Not Found}

The listed URL at repository/commits/statuses is not a valid GitLab API endpoint. I believe it should be repository/commits/:sha/statuses.

spencerschrock commented 7 months ago

We don't actually set the endpoint ourselves, we use a library. https://github.com/ossf/scorecard/blob/b3ad602a5940f097a21601a78dd06a867ddfa46a/clients/gitlabrepo/statuses.go#L35-L43

The problem seems to be that pr.HeadSHA isn't set (correctly) here: https://github.com/ossf/scorecard/blob/b3ad602a5940f097a21601a78dd06a867ddfa46a/checks/raw/ci_tests.go#L61-L66

Based on the graphQL documentation: https://docs.gitlab.com/ee/api/graphql/reference/#mergerequest: In the graphQL query we fetch mergeCommitSha which is "SHA of the merge request commit (set once merged)." https://github.com/ossf/scorecard/blob/b3ad602a5940f097a21601a78dd06a867ddfa46a/clients/gitlabrepo/graphql.go#L64-L96

I assume we actually want diffHeadSha which is "Diff head SHA of the merge request."

aarongoldenthal commented 7 months ago

After doing a few tests with the GraphQL API, yes diffHeadSha looks to be the correct field. The mergeCommitSha field returns null for the following cases:

spencerschrock commented 7 months ago

yes diffHeadSha looks to be the correct field. The mergeCommitSha field returns null for the following cases:

  • MR is open
  • MR is closed
  • MR was merged, but the merge method was fast-forward merge (so no merge commit was created)

Thanks for the info, we've run into issues with different merge styles before, so knowing about the fast-forward merge for GitLab is helpful. Aside from being null or not, I think it's also the wrong semantic. We want the head SHA from while it was still a PR so we can determine if CI-Tests ran before it was merged. So diffHeadSha would help match the semantics our GitHub client already does.

After doing a few tests with the GraphQL API

If this experimentation was using the Scorecard codebase, or you're interested, feel free to send a PR. Or someone else can get to it