search

SiriusStarr / elm-review-no-unsorted

elm-review rules to ensure sortable code is sorted in the "proper" order
GNU General Public License v3.0
3 stars 1 forks source link

`NoUnsortedRecords` leads to out-of-memory crash #15

Open ahankinson opened 1 month ago

ahankinson commented 1 month ago

When enabling NoUnsortedRecords in my elm review rules, the process crashes with an out-of-memory error. Disabling this rule allows elm-review to work. 

node: 22.8.0 / 22.9.0
elm-review --version == 2.12.0
"jfmengels/elm-review": "2.14.0"
"SiriusStarr/elm-review-no-unsorted": "1.1.8"

Macbook Pro, M2 Pro, 32GB, macOS Sequoia 15.0 (although it also crashed in previous versions too).

Statistics of project being reviewed: 

$ tokei
===============================================================================
 Language            Files        Lines         Code     Comments       Blanks
===============================================================================
 Elm                   187        27812        23272          621         3919
===============================================================================
 Total                 187        27812        23272          621         3919
===============================================================================
Stack Trace ``` <--- Last few GCs ---> [73294:0x120008000] 14114 ms: Mark-Compact 4062.9 (4136.3) -> 4062.1 (4135.6) MB, pooled: 1 MB, 544.75 / 0.00 ms (average mu = 0.647, current mu = 0.091) allocation failure; scavenge might not succeed [73294:0x120008000] 14146 ms: Scavenge (interleaved) 4070.0 (4135.6) -> 4070.0 (4159.6) MB, pooled: 0 MB, 19.46 / 0.00 ms (average mu = 0.647, current mu = 0.091) allocation failure; <--- JS stacktrace ---> FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory ----- Native stack trace ----- 1: 0x10255b730 node::OOMErrorHandler(char const*, v8::OOMDetails const&) [/opt/homebrew/Cellar/node/22.8.0/bin/node] 2: 0x102702eb4 v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, v8::OOMDetails const&) [/opt/homebrew/Cellar/node/22.8.0/bin/node] 3: 0x102702e68 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, v8::OOMDetails const&) [/opt/homebrew/Cellar/node/22.8.0/bin/node] 4: 0x1028abdc0 v8::internal::Heap::CallGCPrologueCallbacks(v8::GCType, v8::GCCallbackFlags, v8::internal::GCTracer::Scope::ScopeId) [/opt/homebrew/Cellar/node/22.8.0/bin/node] 5: 0x1028aac3c v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [/opt/homebrew/Cellar/node/22.8.0/bin/node] 6: 0x1028a1104 v8::internal::HeapAllocator::AllocateRawWithLightRetrySlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [/opt/homebrew/Cellar/node/22.8.0/bin/node] 7: 0x1028a1870 v8::internal::HeapAllocator::AllocateRawWithRetryOrFailSlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [/opt/homebrew/Cellar/node/22.8.0/bin/node] 8: 0x102878428 v8::internal::FactoryBase::AllocateRawArray(int, v8::internal::AllocationType) [/opt/homebrew/Cellar/node/22.8.0/bin/node] 9: 0x1028786f8 v8::internal::FactoryBase::NewFixedArrayWithFiller(v8::internal::Handle, int, v8::internal::Handle, v8::internal::AllocationType) [/opt/homebrew/Cellar/node/22.8.0/bin/node] 10: 0x102ac1ddc v8::internal::Handle v8::internal::HashTable>::New(v8::internal::Isolate*, int, v8::internal::AllocationType, v8::internal::MinimumCapacity) [/opt/homebrew/Cellar/node/22.8.0/bin/node] 11: 0x102972eac v8::internal::MaybeHandle v8::internal::JsonParser::ParseJsonValue(v8::internal::Handle) [/opt/homebrew/Cellar/node/22.8.0/bin/node] 12: 0x10297125c v8::internal::JsonParser::ParseJson(v8::internal::Handle) [/opt/homebrew/Cellar/node/22.8.0/bin/node] 13: 0x102971190 v8::internal::JsonParser::Parse(v8::internal::Isolate*, v8::internal::Handle, v8::internal::Handle) [/opt/homebrew/Cellar/node/22.8.0/bin/node] 14: 0x102761470 v8::internal::Builtin_JsonParse(int, unsigned long*, v8::internal::Isolate*) [/opt/homebrew/Cellar/node/22.8.0/bin/node] 15: 0x102349b94 Builtins_CEntry_Return1_ArgvOnStack_BuiltinExit [/opt/homebrew/Cellar/node/22.8.0/bin/node] 16: 0x10b31708c 17: 0x1022f1410 Builtins_AsyncFunctionAwaitResolveClosure [/opt/homebrew/Cellar/node/22.8.0/bin/node] 18: 0x1023bc658 Builtins_PromiseFulfillReactionJob [/opt/homebrew/Cellar/node/22.8.0/bin/node] 19: 0x1022e1714 Builtins_RunMicrotasks [/opt/homebrew/Cellar/node/22.8.0/bin/node] 20: 0x1022b2af4 Builtins_JSRunMicrotasksEntry [/opt/homebrew/Cellar/node/22.8.0/bin/node] 21: 0x10281663c v8::internal::(anonymous namespace)::Invoke(v8::internal::Isolate*, v8::internal::(anonymous namespace)::InvokeParams const&) [/opt/homebrew/Cellar/node/22.8.0/bin/node] 22: 0x102816d44 v8::internal::(anonymous namespace)::InvokeWithTryCatch(v8::internal::Isolate*, v8::internal::(anonymous namespace)::InvokeParams const&) [/opt/homebrew/Cellar/node/22.8.0/bin/node] 23: 0x10283a550 v8::internal::MicrotaskQueue::RunMicrotasks(v8::internal::Isolate*) [/opt/homebrew/Cellar/node/22.8.0/bin/node] 24: 0x10283a2e0 v8::internal::MicrotaskQueue::PerformCheckpointInternal(v8::Isolate*) [/opt/homebrew/Cellar/node/22.8.0/bin/node] 25: 0x1024695e8 node::InternalCallbackScope::Close() [/opt/homebrew/Cellar/node/22.8.0/bin/node] 26: 0x102469b68 node::InternalMakeCallback(node::Environment*, v8::Local, v8::Local, v8::Local, int, v8::Local*, node::async_context, v8::Local) [/opt/homebrew/Cellar/node/22.8.0/bin/node] 27: 0x102481a78 node::AsyncWrap::MakeCallback(v8::Local, int, v8::Local*) [/opt/homebrew/Cellar/node/22.8.0/bin/node] 28: 0x102561c50 node::fs::FSReqCallback::Resolve(v8::Local) [/opt/homebrew/Cellar/node/22.8.0/bin/node] 29: 0x1025635f0 node::fs::AfterNoArgs(uv_fs_s*) [/opt/homebrew/Cellar/node/22.8.0/bin/node] 30: 0x102552868 node::MakeLibuvRequestCallback::Wrapper(uv_fs_s*) [/opt/homebrew/Cellar/node/22.8.0/bin/node] 31: 0x10678edfc uv__work_done [/opt/homebrew/Cellar/libuv/1.48.0/lib/libuv.1.dylib] 32: 0x1067924a8 uv__async_io [/opt/homebrew/Cellar/libuv/1.48.0/lib/libuv.1.dylib] 33: 0x1067a2164 uv__io_poll [/opt/homebrew/Cellar/libuv/1.48.0/lib/libuv.1.dylib] 34: 0x10679293c uv_run [/opt/homebrew/Cellar/libuv/1.48.0/lib/libuv.1.dylib] 35: 0x10246a42c node::SpinEventLoopInternal(node::Environment*) [/opt/homebrew/Cellar/node/22.8.0/bin/node] 36: 0x1025a9280 node::NodeMainInstance::Run(node::ExitCode*, node::Environment*) [/opt/homebrew/Cellar/node/22.8.0/bin/node] 37: 0x1025a8fd4 node::NodeMainInstance::Run() [/opt/homebrew/Cellar/node/22.8.0/bin/node] 38: 0x1025172b8 node::Start(int, char**) [/opt/homebrew/Cellar/node/22.8.0/bin/node] 39: 0x19f71c274 start [/usr/lib/dyld] error Command failed with signal "SIGABRT". ```
SiriusStarr commented 1 month ago

Can you please see if you can reproduce this with v1.1.7?

elm-review --template SiriusStarr/elm-review-no-unsorted/example#1.1.7 --rules NoUnsortedRecords

(And run the 1.1.8 example as well just to compare like to like.)

ahankinson commented 1 month ago

It's a bit tricky to track down.

I can get 1.1.8 to run if I remove the "generated code" folder in .elm-stuff and re-run it on the project. If I try to re-run, It runs out of memory. This is with the '--template' just running this rule.

I also experience the same thing with 1.1.7 changing the template version.

However, if I install / uninstall the modules with elm-json, I can get 1.1.7 to run pretty consistently, but 1.1.8 crashes.

It's kinda like the #1.1.7 specifier doesn't work and that it's always running 1.1.8 with the --template argument?

Anyway, I think I can say that, when running the specific version in my project, 1.1.7 works fine, and 1.1.8 crashes.

SiriusStarr commented 1 month ago

I don't know that this is really "fixable" until elm-review gets type information. NoUnsortedRecords basically has to store info on every single top level declaration in your entire project (and all its dependencies) to try to make sense of the types of record fields. That could obviously be turned off, at the cost of records just becoming ambiguous if they have the same field names but different types.

As a workaround, you should be able to just increase the heap allocation limit of node. Can you try

NODE_OPTIONS=--max_old_space_size=4096 npx elm-review --template SiriusStarr/elm-review-no-unsorted/example#1.1.8 --rules NoUnsortedRecords

or the like (I think the default is only like 1.4 GB or something) and see if that works fine for you? I don't think this is a bug in the rule so much as just running up into the default limits of the node runtime.