Pull requests scans failing as of semgrep version 0.101.0 on git checkout

[arunkumar-telnyx]

Hi @spencerdrak As per https://github.com/returntocorp/semgrep/issues/5562

I am also facing the same issue on most of the repos now with **git checkout**

using

name: Semgrep
on:
  pull_request: {}
  push:
    branches:
      - master
      - main
    paths:
      - .github/workflows/semgrep.yml
  schedule:
    - cron: '25 1 * * 4'
jobs:
  semgrep:
    name: Static Analysis Scan
    runs-on: 2xsmall
    env:
      SEMGREP_APP_TOKEN: ${{ secrets.SEMGREP_APP_TOKEN }}
    container:
      image: returntocorp/semgrep
    if: (github.actor != 'dependabot[bot]')
    steps:
      - uses: actions/checkout@v3
      - run: semgrep ci

Error

Run semgrep ci
Scan environment:
  versions    - semgrep 0.101.0 on python 3.10.5
  environment - running in environment github-actions, triggering event is pull_request
  semgrep.dev - authenticated as team
Fixing git state for github action pull request
Not on head ref: 0a2a0ad2ea131c4[6](https://github.com/team/infra/runs/7086714865?check_suite_focus=true#step:4:7)95[7](https://github.com/team/infra/runs//7086714865?check_suite_focus=true#step:4:8)c75d2[8](https://github.com/team/infra/runs//7086714865?check_suite_focus=true#step:4:9)35c6bc[9](https://github.com/team/infra/runs//7086714865?check_suite_focus=true#step:4:10)3b422869; checking that out now.
Command '['git', 'checkout', '0a2a0ad2ea131c46957c75d2835c6bc93b422869']' returned non-zero exit status 128.
Traceback (most recent call last):
  File "/usr/local/lib/python3.[10](https://github.com/team/infra/runs/7086714865?check_suite_focus=true#step:4:11)/site-packages/semgrep/commands/wrapper.py", line 35, in wrapper
    func(*args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/semgrep/commands/ci.py", line 306, in ci
    with fix_head_if_github_action(metadata):
  File "/usr/local/lib/python3.10/contextlib.py", line 135, in __enter__
    return next(self.gen)
  File "/usr/local/lib/python3.10/site-packages/semgrep/commands/ci.py", line 104, in fix_head_if_github_action
    checkout = subprocess.run(
  File "/usr/local/lib/python3.10/subprocess.py", line 524, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['git', 'checkout', '0a2a0ad2ea131c46957c75d2835c6bc93b422869']' returned non-zero exit status [12](https://github.com/team/infra/runs/7086714865?check_suite_focus=true#step:4:13)8.
Error: Process completed with exit code 2.

Can this be fixed quicker as we have a lot of repos integrated to semgrep and it will start failing on 1000s of repos.

[r2c-demo]

This issue is synced in Linear at https://linear.app/r2c/issue/PA-1577/pull-requests-scans-failing-as-of-semgrep-version-01010-on-git. Note: this link is for r2c use only and is not accessible publicly.

[invidian]

Perhaps someone can explain me, why on earth semgrep needs to do anything Git-related when running on CI? When scanning full repository, shouldn't it not care about git at all and just go over the files?

[segevfiner]

It's this piece of code as far as I can tell, there is a doc comment explaining why it does that, but it obviously doesn't work correctly currently: https://github.com/returntocorp/semgrep/blob/3548a568b7fe4661e045237e315f048c11072209/cli/src/semgrep/commands/ci.py#L74

[spencerdrak]

Thanks for the report - we're looking into this and will leave updates here.

[IagoAbal]

@arunkumar-telnyx can you revert to 0.100.0 in the meantime?

[arunkumar-telnyx]

@IagoAbal It takes the latest version by default while running the workflow . Also, there are around 1000's of repos that are integrated.

[spencerdrak]

we've rolled back our docker image to the latest version, so if you're relying on that tag, then you should be back in operation. We're continuing a deeper dive into the underlying problem and will push a fix when we find out what went wrong. Stay tuned for updates from @IagoAbal or I. Thanks!

[arunkumar-telnyx]

Hi @spencerdrak

Scan environment:
  versions    - semgrep 0.100.0 on python 3.10.5
  environment - running in environment github-actions, triggering event is pull_request
  semgrep.dev - authenticated as team-telnyx
Command '['git', 'fetch', 'origin', '--depth=1', '--force', '--update-head-ok', 'INFRA-9459-vault-update:INFRA-9459-vault-update']' returned non-zero exit status 12[8](https://github.com/team-telnyx/infra-proxy-vault-ha-dev/runs/7097111796?check_suite_focus=true#step:4:9).
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/semgrep/commands/wrapper.py", line 35, in wrapper
    func(*args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/semgrep/commands/ci.py", line 306, in ci
    with fix_head_if_github_action(metadata):
  File "/usr/local/lib/python3.10/contextlib.py", line 135, in __enter__
    return next(self.gen)
  File "/usr/local/lib/python3.10/site-packages/semgrep/commands/ci.py", line 83, in fix_head_if_github_action
    assert metadata.head_branch_hash is not None  # Not none when github action PR
  File "/usr/local/lib/python3.10/site-packages/boltons/cacheutils.py", line 641, in __get__
    value = obj.__dict__[self.func.__name__] = self.func(obj)
  File "/usr/local/lib/python3.10/site-packages/semgrep/meta.py", line 250, in head_branch_hash
    commit = self._get_latest_commit_hash_in_branch(head_branch_name)
  File "/usr/local/lib/python3.10/site-packages/semgrep/meta.py", line 238, in _get_latest_commit_hash_in_branch
    self._shallow_fetch_branch(branch_name)
  File "/usr/local/lib/python3.10/site-packages/semgrep/meta.py", line 215, in _shallow_fetch_branch
    subprocess.run(
  File "/usr/local/lib/python3.10/subprocess.py", line 524, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['git', 'fetch', 'origin', '--depth=1', '--force', '--update-head-ok', 'INFRA-[9](https://github.com/team-telnyx/infra-proxy-vault-ha-dev/runs/7097111796?check_suite_focus=true#step:4:10)459-vault-update:INFRA-9459-vault-update']' returned non-zero exit status [12](https://github.com/team-telnyx/infra-proxy-vault-ha-dev/runs/7097111796?check_suite_focus=true#step:4:13)8.
Error: Process completed with exit code 2.

got it for git fetch, can you revert to older version than this ?

[spencerdrak]

We've identified and corrected the issue here, we're releasing a bugfix version which should take care of the issue. If anything else goes wrong, please feel to let us know (either here or by opening a new ticket).

[IagoAbal]

We released 0.101.1 that should fix both issues, and this version should already be running on App. Could you confirm it works for you now @arunkumar-telnyx ?

[arunkumar-telnyx]

@IagoAbal and @spencerdrak It has been fixed now. Thanks :)