search

digital-go-jp / abr-geocoder

Address Base Registry Geocoder by digital.go.jp
https://lp.geocoder.address-br.digital.go.jp/
MIT License
64 stars 24 forks source link

メモリリーク #184

Open maskatsum opened 1 week ago

maskatsum commented 1 week ago

100万件前後で Out Of Memory になる。今のところ根本的な解決策がない（暫定的な解決策のアイディアはある） 今後対応していく。

 ██░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░ 5% | 1166148/22500835 | geocoding... | ETA: 6h40m
<--- Last few GCs --->

[48649:0x138008000]  1004924 ms: Scavenge 4026.1 (4126.0) -> 4022.9 (4126.0) MB, 11.50 / 0.00 ms  (average mu = 0.913, current mu = 0.866) allocation failure; 
[48649:0x138008000]  1004965 ms: Scavenge 4040.5 (4140.0) -> 4037.3 (4140.0) MB, 10.25 / 0.00 ms  (average mu = 0.913, current mu = 0.866) allocation failure; 
[48649:0x138008000]  1004999 ms: Scavenge 4040.8 (4140.0) -> 4037.7 (4148.0) MB, 10.38 / 0.00 ms  (average mu = 0.913, current mu = 0.866) allocation failure; 

<--- JS stacktrace --->

FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory
----- Native stack trace -----

 1: 0x1044135a4 node::OOMErrorHandler(char const*, v8::OOMDetails const&) [/Users/maskatsum/.volta/tools/image/node/20.17.0/bin/node]
 2: 0x10459c04c v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, v8::OOMDetails const&) [/Users/maskatsum/.volta/tools/image/node/20.17.0/bin/node]
 3: 0x104770720 v8::internal::Heap::GarbageCollectionReasonToString(v8::internal::GarbageCollectionReason) [/Users/maskatsum/.volta/tools/image/node/20.17.0/bin/node]
 4: 0x10476f1fc v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [/Users/maskatsum/.volta/tools/image/node/20.17.0/bin/node]
 5: 0x104765a14 v8::internal::HeapAllocator::AllocateRawWithLightRetrySlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [/Users/maskatsum/.volta/tools/image/node/20.17.0/bin/node]
 6: 0x104766274 v8::internal::HeapAllocator::AllocateRawWithRetryOrFailSlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [/Users/maskatsum/.volta/tools/image/node/20.17.0/bin/node]
 7: 0x10474b2e4 v8::internal::Factory::NewFillerObject(int, v8::internal::AllocationAlignment, v8::internal::AllocationType, v8::internal::AllocationOrigin) [/Users/maskatsum/.volta/tools/image/node/20.17.0/bin/node]
 8: 0x104b32ff4 v8::internal::Runtime_AllocateInYoungGeneration(int, unsigned long*, v8::internal::Isolate*) [/Users/maskatsum/.volta/tools/image/node/20.17.0/bin/node]
 9: 0x104e90c44 Builtins_CEntry_Return1_ArgvOnStack_NoBuiltinExit [/Users/maskatsum/.volta/tools/image/node/20.17.0/bin/node]
10: 0x10a013944 
11: 0x10a0172b4 
12: 0x104e3f210 Builtins_AsyncFunctionAwaitResolveClosure [/Users/maskatsum/.volta/tools/image/node/20.17.0/bin/node]
13: 0x104eecfb8 Builtins_PromiseFulfillReactionJob [/Users/maskatsum/.volta/tools/image/node/20.17.0/bin/node]
14: 0x104e2eb94 Builtins_RunMicrotasks [/Users/maskatsum/.volta/tools/image/node/20.17.0/bin/node]
15: 0x104e063f4 Builtins_JSRunMicrotasksEntry [/Users/maskatsum/.volta/tools/image/node/20.17.0/bin/node]
16: 0x1046ddba0 v8::internal::(anonymous namespace)::Invoke(v8::internal::Isolate*, v8::internal::(anonymous namespace)::InvokeParams const&) [/Users/maskatsum/.volta/tools/image/node/20.17.0/bin/node]
17: 0x1046de08c v8::internal::(anonymous namespace)::InvokeWithTryCatch(v8::internal::Isolate*, v8::internal::(anonymous namespace)::InvokeParams const&) [/Users/maskatsum/.volta/tools/image/node/20.17.0/bin/node]
18: 0x1046de268 v8::internal::Execution::TryRunMicrotasks(v8::internal::Isolate*, v8::internal::MicrotaskQueue*) [/Users/maskatsum/.volta/tools/image/node/20.17.0/bin/node]
19: 0x104705434 v8::internal::MicrotaskQueue::RunMicrotasks(v8::internal::Isolate*) [/Users/maskatsum/.volta/tools/image/node/20.17.0/bin/node]
20: 0x104705bd0 v8::internal::MicrotaskQueue::PerformCheckpoint(v8::Isolate*) [/Users/maskatsum/.volta/tools/image/node/20.17.0/bin/node]
21: 0x104e09ef0 Builtins_CallApiCallback [/Users/maskatsum/.volta/tools/image/node/20.17.0/bin/node]
22: 0x109f382e0 
23: 0x104e0650c Builtins_JSEntryTrampoline [/Users/maskatsum/.volta/tools/image/node/20.17.0/bin/node]
24: 0x104e061f4 Builtins_JSEntry [/Users/maskatsum/.volta/tools/image/node/20.17.0/bin/node]
25: 0x1046ddbc8 v8::internal::(anonymous namespace)::Invoke(v8::internal::Isolate*, v8::internal::(anonymous namespace)::InvokeParams const&) [/Users/maskatsum/.volta/tools/image/node/20.17.0/bin/node]
26: 0x1046dd014 v8::internal::Execution::Call(v8::internal::Isolate*, v8::internal::Handle<v8::internal::Object>, v8::internal::Handle<v8::internal::Object>, int, v8::internal::Handle<v8::internal::Object>*) [/Users/maskatsum/.volta/tools/image/node/20.17.0/bin/node]
27: 0x1045b7904 v8::Function::Call(v8::Local<v8::Context>, v8::Local<v8::Value>, int, v8::Local<v8::Value>*) [/Users/maskatsum/.volta/tools/image/node/20.17.0/bin/node]
28: 0x10433cd24 node::InternalCallbackScope::Close() [/Users/maskatsum/.volta/tools/image/node/20.17.0/bin/node]
29: 0x10433cfe4 node::InternalMakeCallback(node::Environment*, v8::Local<v8::Object>, v8::Local<v8::Object>, v8::Local<v8::Function>, int, v8::Local<v8::Value>*, node::async_context) [/Users/maskatsum/.volta/tools/image/node/20.17.0/bin/node]
30: 0x104353068 node::AsyncWrap::MakeCallback(v8::Local<v8::Function>, int, v8::Local<v8::Value>*) [/Users/maskatsum/.volta/tools/image/node/20.17.0/bin/node]
31: 0x104418d80 node::fs::FSReqCallback::Resolve(v8::Local<v8::Value>) [/Users/maskatsum/.volta/tools/image/node/20.17.0/bin/node]
32: 0x104419a88 node::fs::AfterInteger(uv_fs_s*) [/Users/maskatsum/.volta/tools/image/node/20.17.0/bin/node]
33: 0x10440d430 node::MakeLibuvRequestCallback<uv_fs_s, void (*)(uv_fs_s*)>::Wrapper(uv_fs_s*) [/Users/maskatsum/.volta/tools/image/node/20.17.0/bin/node]
34: 0x104de3b00 uv__work_done [/Users/maskatsum/.volta/tools/image/node/20.17.0/bin/node]
35: 0x104de7550 uv__async_io [/Users/maskatsum/.volta/tools/image/node/20.17.0/bin/node]
36: 0x104df9628 uv__io_poll [/Users/maskatsum/.volta/tools/image/node/20.17.0/bin/node]
37: 0x104de7b14 uv_run [/Users/maskatsum/.volta/tools/image/node/20.17.0/bin/node]
38: 0x10433d6f0 node::SpinEventLoopInternal(node::Environment*) [/Users/maskatsum/.volta/tools/image/node/20.17.0/bin/node]
39: 0x1044533f0 node::NodeMainInstance::Run(node::ExitCode*, node::Environment*) [/Users/maskatsum/.volta/tools/image/node/20.17.0/bin/node]
40: 0x104453104 node::NodeMainInstance::Run() [/Users/maskatsum/.volta/tools/image/node/20.17.0/bin/node]
41: 0x1043dafa0 node::Start(int, char**) [/Users/maskatsum/.volta/tools/image/node/20.17.0/bin/node]
42: 0x195f60274 start [/usr/lib/dyld]
maskatsum commented 5 days ago

解決の目処が立ってきました。