"Code scanning cannot determine the alerts introduced or fixed by this pull request"

[rfay]

Recently I see failures always from codeql, "Code scanning cannot determine the alerts introduced or fixed by this pull request"

Although I have only the go language configured:

    strategy:
      fail-fast: true
      matrix:
        language: [ 'go' ]

(See https://github.com/drud/ddev/blob/a5edc5dde1a630658d7acfded2bfd11b53811f25/.github/workflows/codeql.yml#L32-L37)

it still complains that it can't find a python run.

Is this because it's a forked PR perhaps? Anyway, it's pretty unuseful behavior.

Result is in https://github.com/drud/ddev/pull/4118/checks?check_run_id=7822079521

[aeisenberg]

Early on, there was an analysis for python. See https://github.com/drud/ddev/commit/112ea75ac00ae2ba06cbfc412d08084302867a76. The way that code scanning works is that it compares the latest analysis for the branch you are merging into with the analysis for the PR you are running (for each language). Code scanning then compares the two analyses to determine which alerts are new, fixed, or already existing.

Since there was once an analysis uploaded for python (presumably by accident) into the main branc, code scanning is looking for a python analysis in your PR, but it's not finding one.

The simplest thing to do is just delete the python analysis from your repo. See the GitHub API: https://docs.github.com/en/rest/code-scanning#delete-a-code-scanning-analysis-from-a-repository

[rfay]

Thanks. It's not clear to me how to know the ANALYSIS_ID that needs to be deleted.

Or did you perhaps delete it? I don't see this happening now.

I used gh api -H "Accept: application/vnd.github+json" /repos/drud/ddev/code-scanning/analyses |jq | less to see any analyses that had python... but there are none now, see gist

Originally the action had auto-detected python usage, and yes I had turned that off because it wasn't useful on this project.

[aeisenberg]

Hmmm...I didn't do anything. Your Code Scanning results pages are still showing a missing python analysis. I am not sure why it is not showing up when you list all the analyses. Let me ask internally.

[aeisenberg]

Ah...the python analyses do indeed exist, you just need to paginate your request. Try this instead:

gh api -H "Accept: application/vnd.github+json" /repos/drud/ddev/code-scanning/analyses --paginate | jq | less

And you'll see the analyses for python. Something like this will work:

gh api -H "Accept: application/vnd.github+json" -X delete /repos/drud/ddev/code-scanning/analyses -F "ref=refs/heads/master" -F "tool=CodeQL" -F "analysis_key=github/workflows/codeql.yml:analyze" -F "environment={\"language\":\"python\"}"

This will delete the most recent analysis for python on the main branch. See the docs for how to delete all of the analyses.

[rfay]

gh api -H "Accept: application/vnd.github+json" -X delete /repos/drud/ddev/code-scanning/analyses -F "ref=refs/heads/master" -F "tool=CodeQL" -F "analysis_key=github/workflows/codeql.yml:analyze" -F "environment={\"language\":\"python\"}" gets a 403.

`gh api \

--method DELETE \ -H "Accept: application/vnd.github+json" \ /repos/drud/ddev/code-scanning/analyses/31312199`

(using the python analysis found with gh api -H "Accept: application/vnd.github+json" /repos/drud/ddev/code-scanning/analyses --paginate | jq | less almost works, but gets gh: Analysis is last of its type and deletion may result in the loss of historical alert data. Please specify confirm_delete. (HTTP 400)

"Analysis is last of its type and deletion may result in the loss of historical alert data. Please specify confirm_delete"

How do I specify confim_delete ? :) I see it as a query parameter in https://docs.github.com/en/rest/code-scanning#delete-a-code-scanning-analysis-from-a-repository but there's no example how to use it.

I tried gh api --method DELETE -H "Accept: application/vnd.github+json" --field confirm_delete=true /repos/drud/ddev/code-scanning/analyses/31312199 but without success, same message gh: Analysis is last of its type and deletion may result in the loss of historical alert data. Please specify confirm_delete. (HTTP 400)

[aeisenberg]

Not sure why that isn't working for you. Maybe try the curl variant of the API (just replace <TOKEN> with your token:

curl \
  -X DELETE \
  -H "Accept: application/vnd.github+json" \ 
  -H "Authorization: token <TOKEN>" \
  https://api.github.com/repos/drud/ddev/code-scanning/analyses/31312199?confirm_delete

[rfay]

Well, I used the curl to get rid of that one, but there are more, and

curl -X DELETE -H "Accept: application/vnd.github+json" -H "Authorization: token $GITHUB_TOKEN" https://api.github.com/repos/drud/ddev/code-scanning/analyses/31308286?confirm_delete

gets

{
  "message": "Analysis specified is not deletable.",
  "documentation_url": "https://docs.github.com/rest/reference/code-scanning#delete-a-code-scanning-analysis-from-a-repository"
}

I see that that one has "deletable": false,:

  {
    "ref": "refs/heads/master",
    "commit_sha": "125f700de5ae37d9686359a4a6bff4aea9b0f2a3",
    "analysis_key": ".github/workflows/codeql.yml:analyze",
    "environment": "{\"language\":\"python\"}",
    "category": ".github/workflows/codeql.yml:analyze/language:python",
    "error": "",
    "created_at": "2022-06-03T23:27:27Z",
    "results_count": 0,
    "rules_count": 34,
    "id": 31308286,
    "url": "https://api.github.com/repos/drud/ddev/code-scanning/analyses/31308286",
    "sarif_id": "bad68c30-e394-11ec-90b4-161f2c5671f5",
    "tool": {
      "name": "CodeQL",
      "guid": null,
      "version": "2.9.2"
    },
    "deletable": false,
    "warning": ""
  },

In fact, there are several more python items that are not deletable, and there are 16,900 of these python ones still out there.

Is this a lost cause? Is there any way to wipe it all out and start over? I thought probably codeql had some value but am not seeing it at this point.

[aeisenberg]

I apologize that you are having so much difficulty with this and I understand your frustration. Managing and deleting old analyses is something we are discussing internally to make this process easier. I'm discussing internally to see what the best way forward is.

[rfay]

Thanks. I think there's a bug here too... the OP describes misbehavior, not just something I can fix by deleting one analysis, and there's nothing that was done wrong in the use of this action.

So:

• Please fix it so it only compares to the current or recent analyses on the target branch.
• Please fix it so it only compares to analyses that are currently enabled on the target branch.

[thapabishwa]

gh api -X GET -H "Accept: application/vnd.github+json" /repos/xyz-org/abc-repo/code-scanning/analyses | jq '.[].id' | xargs -I {} gh api \
  --method DELETE \
  -H "Accept: application/vnd.github+json" \
  -H "X-GitHub-Api-Version: 2022-11-28" \
"/repos/xyz-org/abc-repo/code-scanning/analyses/{}?confirm_delete"

I ran this command couple of times until this stopped returning code-scanning results

gh api -X GET -H "Accept: application/vnd.github+json" /repos/xyz-org/abc-repo/code-scanning/analyses

[thibaultcha]

Hello!

I feel bad for hijacking this issue, but I am facing a similar problem that we can't seem to get rid of. I remain dumbfounded as to what the problem is exactly.

On our repository (https://github.com/Kong/ngx_wasm_module), we have 2 CI workflows: CI (for PRs and heads) and CI Large (for daily jobs). Both of these workflows have a CodeQL job. You can see these jobs here (CI) and there (CI Large).

All of our jobs in both workflows are complaining because of some CodeQL thing:

---

And the CodeQL view says:

If I click on "View workflow run" for the last CI Large run, it complains about a configuration error, but when I open this run, there seems to be no error whatsoever (this is the last run of this workflow: https://github.com/Kong/ngx_wasm_module/actions/runs/7812348633).

I am not really sure what the problem is, or if we even have more than one problem.

Here is the list of all our scans results with the --paginate option from the GitHub CLI:

scans.json

I tried deleting some of these runs that have "analysis_key": ".github/workflows/ci-large.yml:codeql" using the above GH API commands from @aeisenberg's helpful comment, but even there I just get an HTTP 404 error (even with properly specifying ref=, analysis_key=, and environment=).

Does anybody see what the problem is? I can open a new issue, but since I too like @rfay am facing an issue that I cannot fix with the GitHub UI, I thought I would post this here as well to help moving the needle on this if possible.

Thanks!

[aeisenberg]

@thibaultcha CodeQL analyses are most effectively run in a single, separated workflow. I recommend that you move your codeql runs into a separate workflow that is triggered on pull requests, pushes to your default branch (and any other protected branch), and on a schedule. You can see an example here.

This will ensure that the code scanning back end can properly compare scans on a pull request to the current scan on main. If you run them in separate workflows, then code scanning will give you a warning like you are seeing.

If you have any further problems or questions, please raise a new issue.

[amoraes]

Hi all, in my case the issue was caused by the repo previously having the "dynamic" codeql enabled and then we moved on to using CodeQL cli in our CI/CD tool. I had to delete all the old dynamic analysis to get it back to work and I had multiple repos so I created a bash script:

#!/bin/bash
# This script removes all old dynamic CodeQL reports from a Github repository so it does not cause conflicts with the ones reported by CircleCI
# See https://github.com/github/codeql-action/issues/1179
# Please provide your Github Access Token as an argument to run this script
AUTH=$1
REPOS=()
OWNER="yourorgname"

# Add the repos here
REPOS[1]="repo1"
REPOS[2]="repo2"
REPOS[3]="repo3"

for REPO in "${REPOS[@]}"; do
  echo "Checking repo: ${REPO}"
  # GitHub API endpoint
  API_URL="https://api.github.com/repos/${OWNER}/${REPO}/code-scanning/analyses"

  # Function to fetch and filter results from a specific page
  fetch_and_filter() {
      local page="$1"
      curl -s -L -H "Accept: application/vnd.github+json" -H "Authorization: Bearer $AUTH" -H "X-GitHub-Api-Version: 2022-11-28" "${API_URL}?page=${page}&per_page=100" > out-page${page}

  }

  # Get the first page of results
  fetch_and_filter 1

  # Fetch and filter results from remaining pages
  for ((page=2; page<=100; page++)); do
      fetch_and_filter "${page}"
    LEN=$(cat out-page${page} | jq length)
    if [ $LEN == 0 ]; then
      TOTAL_PAGES=$(expr $page - 1)
      break;
    fi
  done
  echo "Total pages found: $TOTAL_PAGES"
  for ((page=1; page<=$TOTAL_PAGES; page++)); do
    if [ -f out-page${page} ]; then
      readarray -t URLS < <(cat out-page${page} | jq -r '.[] | select(.analysis_key | contains("dynamic")) | select(.deletable == true) | .url')
      for K in "${!URLS[@]}"; do
            URL=${URLS[$K]}
        curl -L \
          -X DELETE \
          -H "Accept: application/vnd.github+json" \
          -H "Authorization: Bearer ${AUTH}" \
          -H "X-GitHub-Api-Version: 2022-11-28" \
            "${URL}?confirm_delete=true"
      done
    fi
  done
  sleep 5
done