search

github / codeql-action

Actions for running CodeQL analysis
MIT License
1.16k stars 322 forks source link

CodeQL Analysis crash with out of memory #1641

Open unional opened 1 year ago

unional commented 1 year ago

https://github.com/unional/async-fp/actions/runs/4653860071/jobs/8235098420

[2023-04-10 02:21:17] [build-stdout] Extracting /home/runner/work/async-fp/async-fp/packages/gizmo/ts/define.define_deps.spec.ts
  [2023-04-10 02:43:15] [build-stderr] <--- Last few GCs --->
  [2023-04-10 02:43:15] [build-stderr] [2032:0x6de5530]  1319634 ms: Mark-sweep 3160.3 (3390.9) -> 3143.5 (3389.8) MB, 2799.8 / 0.0 ms  (average mu = 0.119, current mu = 0.046) allocation failure; scavenge might not succeed
  [2023-04-10 02:43:15] [build-stderr] [2032:0x6de5530]  1319773 ms: Scavenge 3159.3 (3389.8) -> 3144.1 (3389.8) MB, 20.2 / 0.0 ms  (average mu = 0.119, current mu = 0.046) allocation failure; 
  [2023-04-10 02:43:15] [build-stderr] <--- JS stacktrace --->
  [2023-04-10 02:43:15] [build-stderr] FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
  [2023-04-10 02:43:15] [build-stderr]  1: 0xb7b3e0 node::Abort() [node]
  [2023-04-10 02:43:15] [build-stderr]  2: 0xa8c8aa  [node]
  [2023-04-10 02:43:15] [build-stderr]  3: 0xd69100 v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [node]
  [2023-04-10 02:43:15] [build-stderr]  4: 0xd694a7 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [node]
  [2023-04-10 02:43:15] [build-stderr]  5: 0xf46ba5  [node]
  [2023-04-10 02:43:15] [build-stderr]  6: 0xf47aa8 v8::internal::Heap::RecomputeLimits(v8::internal::GarbageCollector) [node]
  [2023-04-10 02:43:15] [build-stderr]  7: 0xf57fb3  [node]
  [2023-04-10 02:43:15] [build-stderr]  8: 0xf58e28 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [node]
  [2023-04-10 02:43:15] [build-stderr]  9: 0xf3378e v8::internal::HeapAllocator::AllocateRawWithLightRetrySlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [node]
  [2023-04-10 02:43:15] [build-stderr] 10: 0xf34b57 v8::internal::HeapAllocator::AllocateRawWithRetryOrFailSlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [node]
  [2023-04-10 02:43:15] [build-stderr] 11: 0xf15d2a v8::internal::Factory::NewFillerObject(int, v8::internal::AllocationAlignment, v8::internal::AllocationType, v8::internal::AllocationOrigin) [node]
  [2023-04-10 02:43:15] [build-stderr] 12: 0x12dacdf v8::internal::Runtime_AllocateInYoungGeneration(int, unsigned long*, v8::internal::Isolate*) [node]
  [2023-04-10 02:43:15] [build-stderr] 13: 0x1707b79  [node]
  [2023-04-10 02:43:16] [build-stderr] Exception while extracting /home/runner/work/async-fp/async-fp/packages/gizmo/ts/define.define_deps.spec.ts.
  [2023-04-10 02:43:16] [build-stderr] The TypeScript parser wrapper crashed, possibly from running out of memory.
  [2023-04-10 02:43:16] [build-stderr]  at com.semmle.ts.extractor.TypeScriptParser.getExceptionFromMalformedResponse(TypeScriptParser.java:413)
  [2023-04-10 02:43:16] [build-stderr]  at com.semmle.ts.extractor.TypeScriptParser.talkToParserWrapper(TypeScriptParser.java:381)
  [2023-04-10 02:43:16] [build-stderr]  at com.semmle.ts.extractor.TypeScriptParser.parse(TypeScriptParser.java:451)
  [2023-04-10 02:43:16] [build-stderr]  at com.semmle.js.extractor.TypeScriptExtractor.extract(TypeScriptExtractor.java:24)
  [2023-04-10 02:43:16] [build-stderr]  at com.semmle.js.extractor.FileExtractor.extractContents(FileExtractor.java:547)
  [2023-04-10 02:43:16] [build-stderr]  at com.semmle.js.extractor.FileExtractor.extract(FileExtractor.java:454)
  [2023-04-10 02:43:16] [build-stderr]  at com.semmle.js.extractor.AutoBuild.doExtract(AutoBuild.java:1234)
  [2023-04-10 02:43:16] [build-stderr]  at com.semmle.js.extractor.AutoBuild.extract(AutoBuild.java:1220)
  [2023-04-10 02:43:16] [build-stderr]  at com.semmle.js.extractor.AutoBuild.extractTypeScriptFiles(AutoBuild.java:1173)
  [2023-04-10 02:43:16] [build-stderr]  at com.semmle.js.extractor.AutoBuild.extractTypeScript(AutoBuild.java:1051)
  [2023-04-10 02:43:16] [build-stderr]  at com.semmle.js.extractor.AutoBuild.extractSource(AutoBuild.java:752)
  [2023-04-10 02:43:16] [build-stderr]  at com.semmle.js.extractor.AutoBuild.run(AutoBuild.java:459)
  [2023-04-10 02:43:16] [build-stderr]  at com.semmle.js.extractor.AutoBuild.main(AutoBuild.java:1329)
  Error: 4-10 02:43:16] [ERROR] Spawned process exited abnormally (code 1; tried to run: [/opt/hostedtoolcache/CodeQL/2.12.6-20230403/x[64](https://github.com/unional/async-fp/actions/runs/4653860071/jobs/8235098420#step:5:65)/codeql/javascript/tools/autobuild.sh])

This tends to happen on TypeScript project.

I have also seen this on type-plus

And a workaround is to ignore the spec files.

unional commented 1 year ago

https://github.com/unional/async-fp/actions/runs/4654029005/jobs/8235410528

[2023-04-10 02:56:10] [build-stdout] Extracting /home/runner/work/async-fp/async-fp/packages/gizmo/ts/define.ts
  [2023-04-10 03:16:38] [build-stderr] <--- Last few GCs --->
  [2023-04-10 03:16:38] [build-stderr] [1970:0x6fc9530]  1228465 ms: Mark-sweep 3160.7 (3392.3) -> 3142.9 (3390.8) MB, 2723.0 / 0.0 ms  (average mu = 0.110, current mu = 0.094) allocation failure; scavenge might not succeed
  [2023-04-10 03:16:38] [build-stderr] [1970:0x6fc9530]  1228604 ms: Scavenge 31[58](https://github.com/unional/async-fp/actions/runs/4654029005/jobs/8235410528#step:5:59).7 (3390.8) -> 3143.5 (3390.8) MB, 24.3 / 0.0 ms  (average mu = 0.110, current mu = 0.094) allocation failure; 
  [2023-04-10 03:16:38] [build-stderr] [1970:0x6fc9530]  1228[69](https://github.com/unional/async-fp/actions/runs/4654029005/jobs/8235410528#step:5:70)9 ms: Scavenge 3158.7 (3390.8) -> 3144.0 (3391.0) MB, 21.4 / 0.0 ms  (average mu = 0.110, current mu = 0.094) allocation failure; 
  [2023-04-10 03:16:38] [build-stderr] <--- JS stacktrace --->
  [2023-04-10 03:16:38] [build-stderr] FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
  [2023-04-10 03:16:38] [build-stderr]  1: 0xb7b3e0 node::Abort() [node]
  [2023-04-10 03:16:38] [build-stderr]  2: 0xa8c8aa  [node]
  [2023-04-10 03:16:38] [build-stderr]  3: 0xd69100 v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [node]
  [2023-04-10 03:16:38] [build-stderr]  4: 0xd694a7 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [node]
  [2023-04-10 03:16:38] [build-stderr]  5: 0xf46ba5  [node]
  [2023-04-10 03:16:38] [build-stderr]  6: 0xf47aa8 v8::internal::Heap::RecomputeLimits(v8::internal::GarbageCollector) [node]
  [2023-04-10 03:16:38] [build-stderr]  7: 0xf57fb3  [node]
  [2023-04-10 03:16:38] [build-stderr]  8: 0xf58e28 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [node]
  [2023-04-10 03:16:38] [build-stderr]  9: 0xf3378e v8::internal::HeapAllocator::AllocateRawWithLightRetrySlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [node]
  [2023-04-10 03:16:38] [build-stderr] 10: 0xf34b57 v8::internal::HeapAllocator::AllocateRawWithRetryOrFailSlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [node]
  [2023-04-10 03:16:38] [build-stderr] 11: 0xf15d2a v8::internal::Factory::NewFillerObject(int, v8::internal::AllocationAlignment, v8::internal::AllocationType, v8::internal::AllocationOrigin) [node]
  [2023-04-10 03:16:38] [build-stderr] 12: 0x12dacdf v8::internal::Runtime_AllocateInYoungGeneration(int, unsigned long*, v8::internal::Isolate*) [node]
  [2023-04-10 03:16:38] [build-stderr] 13: 0x1[70](https://github.com/unional/async-fp/actions/runs/4654029005/jobs/8235410528#step:5:71)7b79  [node]
  [2023-04-10 03:16:38] [build-stderr] Exception while extracting /home/runner/work/async-fp/async-fp/packages/gizmo/ts/define.ts.
  [2023-04-10 03:16:38] [build-stderr] The TypeScript parser wrapper crashed, possibly from running out of memory.

here is a case that the actual code getting out of memory

adityasharad commented 1 year ago

Thanks for reporting. I agree with you that ignoring the spec files is the best workaround, using the paths-ignore property.

We'll take a closer look at the problematic file itself, but in the meantime, please try that workaround to keep your workflow unblocked. Link to the problematic spec file: https://github.com/unional/async-fp/blob/main/packages/gizmo/ts/define.define_deps.spec.ts

unional commented 1 year ago

Thanks, for the last one, the out of memory comes from the actual code. So in this case, I can't ignore it using paths-ignore (well, I can, just meaning I'm not going to check that file(s))

Link to the problematic file: https://github.com/unional/async-fp/blob/main/packages/gizmo/ts/define.ts

The issue is likely caused by analyzing the types, which they are in this file: https://github.com/unional/async-fp/blob/main/packages/gizmo/ts/types.ts

adityasharad commented 1 year ago

Oh yes good point: you will have to ignore all 3 files to avoid the error. Give that a try; I've passed this on to our JavaScript analysis team to investigate the crash itself.

cooljeanius commented 8 months ago

It would be nice if the action printed additional information when it runs out of memory like this (like: what the limit is, how to increase it, or how to reduce your usage to avoid hitting it, etc.)

cooljeanius commented 8 months ago

It would be nice if the action printed additional information when it runs out of memory like this (like: what the limit is, how to increase it, or how to reduce your usage to avoid hitting it, etc.)

Update: it appears that it now does this; @unional if you could try running the job again and seeing if there are any additional messages, that might be helpful