search

github / codeql-action

Actions for running CodeQL analysis
MIT License
1.14k stars 315 forks source link

Perform CodeQL Analysis fails during database finalize due to NoSuchFileException #1338

Open makubacki opened 1 year ago

makubacki commented 1 year ago

While enabling CodeQL, I encounter this error frequently. Occassionally, I do not encounter the error and the GitHub action succeeds.

/opt/hostedtoolcache/CodeQL/0.0.0-20221010/x64/codeql/codeql database finalize --finalize-dataset --threads=2 /home/runner/work/_temp/codeql_databases/cpp --ram=5920
  Running TRAP import for CodeQL database at /home/runner/work/_temp/codeql_databases/cpp...
  A fatal error occurred: Couldn't read /home/runner/work/_temp/codeql_databases/cpp/trap/cpp/source/tmp/ccMtRh�.link
  (eventual cause: NoSuchFileException "/home/runner/work/_temp/codeql_databases/cpp/trap/cpp/source/tmp/ccMtRh�.link")
  A fatal error occurred: Dataset import for /home/runner/work/_temp/codeql_databases/cpp/db-cpp failed with code 2.
  Error: The process '/opt/hostedtoolcache/CodeQL/0.0.0-20221010/x64/codeql/codeql' failed with exit code 2
  Error: The process '/opt/hostedtoolcache/CodeQL/0.0.0-20221010/x64/codeql/codeql' failed with exit code 2
      at toolrunnerErrorCatcher (/home/runner/work/_actions/github/codeql-action/v2/lib/toolrunner-error-catcher.js:86:19)
      at processTicksAndRejections (node:internal/process/task_queues:96:5)
      at async Object.finalizeDatabase (/home/runner/work/_actions/github/codeql-action/v2/lib/codeql.js:[60](https://github.com/tianocore/edk2/actions/runs/3362451486/jobs/5574191203#step:12:61)7:13)
      at async finalizeDatabaseCreation (/home/runner/work/_actions/github/codeql-action/v2/lib/analyze.js:114:13)
      at async runFinalize (/home/runner/work/_actions/github/codeql-action/v2/lib/analyze.js:298:21)
      at async run (/home/runner/work/_actions/github/codeql-action/v2/lib/analyze-action.js:164:29)
      at async runWrapper (/home/runner/work/_actions/github/codeql-action/v2/lib/analyze-action.js:235:9)

The filename that can't be read will change, but the error remains:

A fatal error occurred: Couldn't read /home/runner/work/_temp/codeql_databases/cpp/trap/cpp/source/home/runner/work/edk2/edk2/Build/MdeModule/DEBUG_GCC5/IA32/MdeModulePk�.link

Can you please help provide more information about what is going on?

Erich-McMillan commented 1 year ago

I'm seeing this issue as well, but locally in an ubuntu container:

Finalizing database at /edk2/codeql-cpp-analysis.
A fatal error occurred: Couldn't read /edk2/codeql-cpp-analysis/trap/cpp/source/�♠.link
(eventual cause: NoSuchFileException "/edk2/codeql-cpp-analysis/trap/cpp/source/�♠.link")
Running queries.
A fatal error occurred: /edk2/codeql-cpp-analysis/db-cpp does not seem to be a raw QL dataset; it has no dbscheme.
makubacki commented 1 year ago

We compile multiple "packages" of code (sets of drivers) with a single build command. Building each package in a separate job with a matrix is leading to better results so far.

https://github.com/tianocore/edk2/actions/runs/3366104711

makubacki commented 1 year ago

While it is easier to rerun an individual failed job with the matrix, the same failure does occasionally occur.

jketema commented 1 year ago

Hi @makubacki

Thanks for opening this issue.

Can you please help provide more information about what is going on?

It seems that your build environment is triggering a bug in the CodeQL component ("extractor") responsible for reading your C/C++ source files. We are able to reproduce this behaviour on the GitHub provided runners, but unfortunately not locally. It's also not a failure we have seen before, while this code has not been modified for the last few months.

Could you tell me whether your build environment does anything special with environment variables that are being set (the extractor depends on a few of these, which are set by the action) or whether you somehow override functionality of the system provided glibc library?

makubacki commented 1 year ago

Hi. I triggered a new workflow to periodically dump environment variables here - https://github.com/tianocore/edk2/actions/runs/3388710334. It has a couple failures that reproduce the issue.

I'm not aware of any overrides to glibc.

jketema commented 1 year ago

Thanks. I don't see anything that is particularly of interest in that output.

I'll file an internal issue for this, so we can schedule looking further into this.

makubacki commented 1 year ago

Great, thanks! It looks like a viable workaround for my project is to use Windows builds. So far, I'm seeing reliable results with that. We are still very interested in being able to build on Ubuntu + GCC though.