search

nuxt / bridge

šŸŒ‰ Experience Nuxt 3 features on existing Nuxt 2 projects
MIT License
273 stars 29 forks source link

Memory Leak on AWS ECS #927

Closed antlionguard closed 11 months ago

antlionguard commented 1 year ago

Environment

Reproduction

I cannot reporduce because that's very huge project. I just wanna trace where memory leak happens.

Describe the bug

Here's the memory utilization on AWS ECS Monitoring section. The parts where the decrease occurs are when the container is restarted due to out of memory.

image

Additional context

And how can i trace where memory leaks come from? Any tips for tracing?

Logs

error Command failed with exit code 134.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
Aborted (core dumped)
17: 0x15d9e19 [/usr/local/bin/node]
16: 0x11f9038 v8::internal::Runtime_RegExpExec(int, unsigned long*, v8::internal::Isolate*) [/usr/local/bin/node]
15: 0x11d3e67 v8::internal::RegExpImpl::IrregexpExec(v8::internal::Isolate*, v8::internal::Handle<v8::internal::JSRegExp>, v8::internal::Handle<v8::internal::String>, int, v8::internal::Handle<v8::internal::RegExpMatchInfo>, v8::internal::RegExp::ExecQuirks) [/usr/local/bin/node]
14: 0x11d3cee v8::internal::RegExpImpl::IrregexpPrepare(v8::internal::Isolate*, v8::internal::Handle<v8::internal::JSRegExp>, v8::internal::Handle<v8::internal::String>) [/usr/local/bin/node]
13: 0x11d30f0 v8::internal::RegExpImpl::CompileIrregexp(v8::internal::Isolate*, v8::internal::Handle<v8::internal::JSRegExp>, v8::internal::Handle<v8::internal::String>, bool) [/usr/local/bin/node]
12: 0x11d2947 v8::internal::RegExpImpl::Compile(v8::internal::Isolate*, v8::internal::Zone*, v8::internal::RegExpCompileData*, v8::base::Flags<v8::internal::JSRegExp::Flag, int>, v8::internal::Handle<v8::internal::String>, v8::internal::Handle<v8::internal::String>, bool, unsigned int&) [/usr/local/bin/node]
11: 0x11b45b9 v8::internal::RegExpCompiler::Assemble(v8::internal::Isolate*, v8::internal::RegExpMacroAssembler*, v8::internal::RegExpNode*, int, v8::internal::Handle<v8::internal::String>) [/usr/local/bin/node]
10: 0x14c8d8e v8::internal::RegExpMacroAssemblerX64::GetCode(v8::internal::Handle<v8::internal::String>) [/usr/local/bin/node]
8: 0xe7a215 v8::internal::Factory::CodeBuilder::BuildInternal(bool) [/usr/local/bin/node]
9: 0xe7ac2e v8::internal::Factory::CodeBuilder::Build() [/usr/local/bin/node]
7: 0xe79bb6 v8::internal::Factory::CodeBuilder::AllocateCode(bool) [/usr/local/bin/node]
6: 0xeabe58 v8::internal::Heap::AllocateRawWithRetryOrFailSlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [/usr/local/bin/node]
4: 0xce1d77 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [/usr/local/bin/node]
5: 0xe993e5 [/usr/local/bin/node]
3: 0xce19d0 v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [/usr/local/bin/node]
1: 0xb090e0 node::Abort() [/usr/local/bin/node]
2: 0xa1b70e [/usr/local/bin/node]

<--- Last few GCs --->
[28:0x54111c0] 146309520 ms: Mark-sweep (reduce) 741.0 (771.3) -> 741.0 (767.1) MB, 1014.7 / 0.0 ms (average mu = 0.724, current mu = 0.025) last resort GC in old space requested
[28:0x54111c0] 146310521 ms: Mark-sweep (reduce) 741.0 (767.1) -> 741.0 (766.6) MB, 1001.5 / 0.0 ms (average mu = 0.565, current mu = 0.000) last resort GC in old space requested

<--- JS stacktrace --->

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
danielroe commented 1 year ago

Would you see if you can reproduce with just nitro and, if so, open an issue on https://github.com/unjs/nitro instead? šŸ™

antlionguard commented 1 year ago

Would you see if you can reproduce with just nitro and, if so, open an issue on https://github.com/unjs/nitro instead? šŸ™

I'm not using nitro in this project. Nitro is set false in bridge config. But i'm gonna try with nitro and vite. I hope i can make it. There's lots of error occurs when i set nitro and vite set true.

danielroe commented 1 year ago

Oh, if you're not using nitro then the memory leak will probably still reproduce if you enable it.

What makes you think this is an issue with bridge? Does it reproduce outside AWS ECS?

(I would recommend trying with meta: false btw.)

antlionguard commented 1 year ago

Oh, if you're not using nitro then the memory leak will probably still reproduce if you enable it.

What makes you think this is an issue with bridge? Does it reproduce outside AWS ECS?

(I would recommend trying with meta: false btw.)

Project runs into docker container in ecs. But i cannot reproduce in my local docker enviroment. This makes things quite interesting. When i watching memory usage in node inspect everything is normal. Garbage collector works fine in my local. I'm gonna try with meta: false and I will inform you what will happen next. Thanks for quick replies <3

antlionguard commented 1 year ago

Maybe the reason why I cannot reproduce it in Local env is that this process takes too long. Because when I look at the chart, the process of resetting the container takes approximately 15-20 hours.

image

antlionguard commented 1 year ago

meta: false seems work. I will continue to monitor for 1-2 more days and share the results.

image

antlionguard commented 1 year ago

probably meta: false solved the problem. What can I provide that might help you resolve the issue caused by meta:true? @danielroe

antlionguard commented 11 months ago

There's no memory leak witn latest nuxt-bridge and node 20.10.0.