search

nodejs / node

Node.js JavaScript runtime ✨🐢🚀✨
https://nodejs.org
Other
104.97k stars 28.42k forks source link

Possible memory leak in TLS(Wrap?) #1075

Closed aredridel closed 9 years ago

aredridel commented 9 years ago

I'm diagnosing a problem when I run paypal's npm proxy https://github.com/krakenjs/kappa when run on iojs. After 300 requests for a 50kb tarball, memory allocated grows from 250 mb base to 450 mb.

I've run it under valgrind, and there's some things that look a bit suspicious around tlswrap, smalloc and some other crypto related bits. The interesting bits of the valgrind report are:

==31542== 827,216 bytes in 656 blocks are still reachable in loss record 1,214 of 1,223
==31542==    at 0x4C2B6CD: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==31542==    by 0x7450D8: CRYPTO_malloc (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542==    by 0x6C868B: asn1_enc_save (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542==    by 0x6C2866: ASN1_item_ex_d2i (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542==    by 0x6C14FF: asn1_template_noexp_d2i (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542==    by 0x6C1BAB: asn1_template_ex_d2i (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542==    by 0x6C2F0F: ASN1_item_d2i (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542==    by 0x67CC9E: ssl3_get_server_certificate (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542==    by 0x681323: ssl3_connect (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542==    by 0x67958B: ssl23_connect (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542==    by 0x679E1A: ssl23_write (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542==    by 0xCDBCEF: _ZN4node7TLSWrap7ClearInEv.part.43 (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542== 
==31542== 1,343,488 bytes in 328 blocks are still reachable in loss record 1,215 of 1,223
==31542==    at 0x4C2AC27: operator new[](unsigned long) (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==31542==    by 0xCD825D: node::NodeBIO::PeekWritable(unsigned long*) (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542==    by 0xCD8EEC: node::TLSWrap::OnAllocImpl(unsigned long, uv_buf_t*, void*) (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542==    by 0xD031D0: uv__read (stream.c:1089)
==31542==    by 0xD03967: uv__stream_io (stream.c:1219)
==31542==    by 0xD08CEC: uv__io_poll (linux-core.c:319)
==31542==    by 0xCFA955: uv_run (core.c:324)
==31542==    by 0xC87650: node::Start(int, char**) (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542==    by 0x5C8E76C: (below main) (libc-start.c:226)
==31542== 
==31542== 2,303,504 bytes in 131 blocks are still reachable in loss record 1,216 of 1,223
==31542==    at 0x4C2B6CD: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==31542==    by 0x7450D8: CRYPTO_malloc (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542==    by 0x67BBAF: ssl3_setup_write_buffer (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542==    by 0x67BCB1: ssl3_setup_buffers (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542==    by 0x679000: ssl23_connect (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542==    by 0x679CBA: ssl23_read (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542==    by 0xCDCE78: node::TLSWrap::ClearOut() (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542==    by 0xCDD2B4: node::TLSWrap::Start(v8::FunctionCallbackInfo<v8::Value> const&) (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542==    by 0x801781: v8::internal::FunctionCallbackArguments::Call(void (*)(v8::FunctionCallbackInfo<v8::Value> const&)) (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542==    by 0x828E02: v8::internal::Builtin_HandleApiCall(int, v8::internal::Object**, v8::internal::Isolate*) (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542==    by 0x1C66A79060BA: ???
==31542==    by 0x1C66A7CDC7AD: ???
==31542== 
==31542== 2,323,416 bytes in 131 blocks are still reachable in loss record 1,217 of 1,223
==31542==    at 0x4C2B6CD: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==31542==    by 0x7450D8: CRYPTO_malloc (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542==    by 0x67BD5F: ssl3_setup_buffers (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542==    by 0x679000: ssl23_connect (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542==    by 0x679CBA: ssl23_read (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542==    by 0xCDCE78: node::TLSWrap::ClearOut() (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542==    by 0xCDD2B4: node::TLSWrap::Start(v8::FunctionCallbackInfo<v8::Value> const&) (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542==    by 0x801781: v8::internal::FunctionCallbackArguments::Call(void (*)(v8::FunctionCallbackInfo<v8::Value> const&)) (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542==    by 0x828E02: v8::internal::Builtin_HandleApiCall(int, v8::internal::Object**, v8::internal::Isolate*) (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542==    by 0x1C66A79060BA: ???
==31542==    by 0x1C66A7CDC7AD: ???
==31542==    by 0x1C66A7CDC562: ???
==31542== 
==31542== 3,464,048 bytes in 197 blocks are still reachable in loss record 1,218 of 1,223
==31542==    at 0x4C2B6CD: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==31542==    by 0x7450D8: CRYPTO_malloc (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542==    by 0x67BBAF: ssl3_setup_write_buffer (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542==    by 0x67BCB1: ssl3_setup_buffers (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542==    by 0x679000: ssl23_connect (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542==    by 0x679CBA: ssl23_read (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542==    by 0xCDCE78: node::TLSWrap::ClearOut() (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542==    by 0xCDD2B4: node::TLSWrap::Start(v8::FunctionCallbackInfo<v8::Value> const&) (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542==    by 0x1C66A7A43345: ???
==31542==    by 0x1C66A7EA169A: ???
==31542==    by 0x1C66A7CDC562: ???
==31542==    by 0x1C66A7924BC5: ???
==31542== 
==31542== 3,493,992 bytes in 197 blocks are still reachable in loss record 1,219 of 1,223
==31542==    at 0x4C2B6CD: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==31542==    by 0x7450D8: CRYPTO_malloc (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542==    by 0x67BD5F: ssl3_setup_buffers (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542==    by 0x679000: ssl23_connect (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542==    by 0x679CBA: ssl23_read (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542==    by 0xCDCE78: node::TLSWrap::ClearOut() (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542==    by 0xCDD2B4: node::TLSWrap::Start(v8::FunctionCallbackInfo<v8::Value> const&) (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542==    by 0x1C66A7A43345: ???
==31542==    by 0x1C66A7EA169A: ???
==31542==    by 0x1C66A7CDC562: ???
==31542==    by 0x1C66A7924BC5: ???
==31542==    by 0x1C66A7C08177: ???
==31542== 
==31542== 5,373,952 bytes in 328 blocks are still reachable in loss record 1,220 of 1,223
==31542==    at 0x4C2AC27: operator new[](unsigned long) (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==31542==    by 0xCD8325: node::NodeBIO::Commit(unsigned long) (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542==    by 0xCDD745: node::TLSWrap::OnReadImpl(long, uv_buf_t const*, uv_handle_type, void*) (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542==    by 0xD0335E: uv__read (stream.c:1156)
==31542==    by 0xD03967: uv__stream_io (stream.c:1219)
==31542==    by 0xD08CEC: uv__io_poll (linux-core.c:319)
==31542==    by 0xCFA955: uv_run (core.c:324)
==31542==    by 0xC87650: node::Start(int, char**) (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542==    by 0x5C8E76C: (below main) (libc-start.c:226)
==31542== 
==31542== 7,422,722 bytes in 131 blocks are still reachable in loss record 1,221 of 1,223
==31542==    at 0x4C2B6CD: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==31542==    by 0xCAD6A2: node::smalloc::Alloc(v8::FunctionCallbackInfo<v8::Value> const&) (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542==    by 0x801781: v8::internal::FunctionCallbackArguments::Call(void (*)(v8::FunctionCallbackInfo<v8::Value> const&)) (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542==    by 0x828E02: v8::internal::Builtin_HandleApiCall(int, v8::internal::Object**, v8::internal::Isolate*) (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542==    by 0x1C66A79060BA: ???
==31542==    by 0x1C66A79B20B8: ???
==31542==    by 0x1C66A791EA54: ???
==31542==    by 0x1C66A791EDA0: ???
==31542==    by 0x1C66A7D088BE: ???
==31542==    by 0x1C66A7D084D4: ???
==31542==    by 0x1C66A7D08272: ???
==31542==    by 0x1C66A7924BC5: ???
==31542== 
==31542== 11,162,414 bytes in 197 blocks are still reachable in loss record 1,222 of 1,223
==31542==    at 0x4C2B6CD: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==31542==    by 0xCAD6A2: node::smalloc::Alloc(v8::FunctionCallbackInfo<v8::Value> const&) (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542==    by 0x801781: v8::internal::FunctionCallbackArguments::Call(void (*)(v8::FunctionCallbackInfo<v8::Value> const&)) (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542==    by 0x828E02: v8::internal::Builtin_HandleApiCall(int, v8::internal::Object**, v8::internal::Isolate*) (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542==    by 0x1C66A79060BA: ???
==31542==    by 0x1C66A79B20B8: ???
==31542==    by 0x1C66A791EA54: ???
==31542==    by 0x1C66A791EDA0: ???
==31542==    by 0x1C66A7EA5607: ???
==31542==    by 0x1C66A7D08272: ???
==31542==    by 0x1C66A7924BC5: ???
==31542==    by 0x1C66A7C08177: ???
==31542== 
==31542== 18,591,935 bytes in 20,677 blocks are still reachable in loss record 1,223 of 1,223
==31542==    at 0x4C2B6CD: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==31542==    by 0xCD8E91: node::TLSWrap::OnAllocSelf(unsigned long, uv_buf_t*, void*) (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542==    by 0xCDCEA0: node::TLSWrap::ClearOut() (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542==    by 0xCDD7B7: node::TLSWrap::OnReadImpl(long, uv_buf_t const*, uv_handle_type, void*) (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542==    by 0xD0335E: uv__read (stream.c:1156)
==31542==    by 0xD03967: uv__stream_io (stream.c:1219)
==31542==    by 0xD08CEC: uv__io_poll (linux-core.c:319)
==31542==    by 0xCFA955: uv_run (core.c:324)
==31542==    by 0xC87650: node::Start(int, char**) (in /home/aredridel/t/node_modules/iojs-bin/node_modules/iojs-linux-x64/iojs-v1.4.2-linux-x64/bin/iojs)
==31542==    by 0x5C8E76C: (below main) (libc-start.c:226)
==31542== 
==31542== LEAK SUMMARY:
==31542==    definitely lost: 10,464 bytes in 327 blocks
==31542==    indirectly lost: 63,532 bytes in 3,258 blocks
==31542==      possibly lost: 4,568 bytes in 64 blocks
==31542==    still reachable: 69,011,541 bytes in 206,575 blocks
==31542==         suppressed: 0 bytes in 0 blocks
==31542== 
==31542== For counts of detected and suppressed errors, rerun with: -v
==31542== ERROR SUMMARY: 42 errors from 42 contexts (suppressed: 2 from 2)

Any help or suggestions for what to look at to track this down would be wonderful.

Node 0.12 and 0.10 don't show this behavior with the same code (we're stress-testing io.js in parallel to shake stuff like this loose)

jasisk commented 9 years ago

Fortunately, the sharp spikes just aren't happening. The smalloc logging seems to show they're being freed. TLSWrap seems to be freed regularly but, lately, seems to have a floor of 600. WriteWrap is more interesting, seemingly alternating between 1 and 590.

RSS is trending in a slightly upward angle, gaining ~10mb over 4 hours, during a time when 0.12 was flat. At this point (couple days of uptime), io.js is ~100mb higher at ~450mb as compared to 0.12's 350mb.

indutny commented 9 years ago

Thank you, @jasisk . Unfortunately, it doesn't seem like anything bad is happening with either of TLSWrap/WriteWrap/smalloc. I can only suggest trying io.js 1.0.0 at this point, to isolate the change that introduced the leak.

jasisk commented 9 years ago

You got it. Will deploy first thing in the morning, report back after a few days.

indutny commented 9 years ago

Thank you, @jasisk!

indutny commented 9 years ago

Any news, @jasisk ?

jasisk commented 9 years ago

I ended up letting the last deploy run over the weekend just to make sure the initial hunch wasn't wrong since it had been a while since I'd let it run for a solid chunk of time.

RSS

Deployed v1.0.0 earlier tonight. We'll see how this goes.

indutny commented 9 years ago

@jasisk I have a fix for you :) Might help a bit, or might fix it... (crossing my fingers).

Going to push it to PR in a bit.

Here are the other changes that needs to be re-checked:

C++

cc @bnoordhuis : if you have time let's iterate this list together and verify that there are no leaks. (I've diffed all changes between v0.12 node.js and iojs v1.1.0 to find these).

indutny commented 9 years ago

@jasisk https://github.com/iojs/io.js/pull/1244 here you go, please give it a try. It is definitely fixing some leak, but it is really hard to tell how much yet.

indutny commented 9 years ago

@jasisk I have very good hopes for https://github.com/iojs/io.js/pull/1244 to fix problem that you are experiencing. (No need to redeploy, the landed commits are just a bit re-iterated patches that you applied in your testing).

jasisk commented 9 years ago

Only 7 hours on it but haven't seen this shape before: RSS

Huge :+1:

balupton commented 9 years ago

@jasisk what are you using to generate those graphs?

jasisk commented 9 years ago

@balupton I'm using Grafana. We stuff all of our metrics into InfluxDB. Looking forward to the upcoming releases of both for various reasons. We also have Heka in the pipeline for metrics-triggered actions. We get graph annotations in Heka, I just need to push those back into Influx because Grafana supports them. twitter@jcse if you want to chat about it.

indutny commented 9 years ago

@jasisk btw, how does the graph look after 2 days?

jasisk commented 9 years ago

Hard to say at this point as we've only got 24 hours on it—I restarted three machines to get a proper comparison. I'm a little concerned by the trend over the last 6 hours but not enough yet to really conclude anything. Though even at this stage I can tell there's been substantial progress.

The build is #1244 on 2db758c562b4e480acc2d1a654e2c471a653a039.

purple: io.js, blue: 0.12.1, yellow: 0.12.0 RSS

indutny commented 9 years ago

@jasisk looks like there is some leak still :) I'll continue looking into it then. Thank you!

indutny commented 9 years ago

@jasisk may I ask you to take a heapsnapshot of the leaking process? I don't need the full file, just a want to take a sneak-peak look at the "Global Handles" contents. (See the attached picture).

screen shot 2015-03-28 at 09 50 52

UPDATE: would be cool to have a bit more items than what present on the attached screenshot :)

jasisk commented 9 years ago

You got it. Instrumented, taking snapshots periodically. Will update when it's grown a bit.

indutny commented 9 years ago

@jasisk how is it going? ;)

jasisk commented 9 years ago

I don't have quite as much data as I'd like (pulled a couple of machines out of the pool for a bit) but, from what I can tell, looks like @rvagg and others' speculation was right: lots of timer refs from stats collection (unref'd timeout).

In any case, I put the machines back, kicked them over and am snapshotting every 10mb.

Fishrock123 commented 9 years ago

@jasisk You mean like https://github.com/joyent/node/issues/8364 / https://github.com/iojs/io.js/pull/1152#issuecomment-88870832?

1152 (the original, but broken fix that is in node 0.12) is unfortunately broken beyond what I can repair. :|

indutny commented 9 years ago

@jasisk no way! haha :) Well, I'd be more than happy if these are just a timers. Do they appear on a Global Handles list?

Fishrock123 commented 9 years ago

See also https://github.com/iojs/io.js/commit/33fea6ed5f506e7748cb9132f63dc03215a9eb39#commitcomment-10440550 for more info on how / why the original #1152 is broken

indutny commented 9 years ago

@Fishrock123 I'm going to address this problem now. Could you please assign it to me if we have any issue in a tracker?

jasisk commented 9 years ago

Yup. That's exactly it.

0.12 on the left, io.js (custom build mentioned above) on the right handles

indutny commented 9 years ago

Yeah, it leaks timers now. I'll figure it out soon using @Fishrock123 patch.

On Thursday, April 2, 2015, Jean-Charles Sisk notifications@github.com wrote:

Yup. That's exactly it.

0.12 on the left, io.js (custom build mentioned above) on the right [image: handles] https://cloud.githubusercontent.com/assets/62923/6972101/6c7f6220-d94f-11e4-8ee8-ef1eb2e85489.png

— Reply to this email directly or view it on GitHub https://github.com/iojs/io.js/issues/1075#issuecomment-89022907.

indutny commented 9 years ago

@jasisk could you please give a try to https://github.com/iojs/io.js/pull/1330 ? I hope it'll help

jasisk commented 9 years ago

:+1: deploying now.

Fishrock123 commented 9 years ago

@jasisk How's it looking? :D

jasisk commented 9 years ago

Hasn't calmed down yet but it's looking really good! I've learned to hold off for a few days—for some reason we've historically seen explosive memory growth on the order of 120mb / day, starting a couple of days in—but can't ask for a better graph so far.

purple: 0.12, blue: 1.x RSS

Fishrock123 commented 9 years ago

@jasisk Mind giving an update? :D

jasisk commented 9 years ago

Was subscribing to, "no news is good news." :grinning:

These two machines are still instrumented to take heap snapshots periodically which is why it's not super clean, however they're acting similarly which makes me think you guys nailed it!

blue: iojs, purple: 0.12 RSS

indutny commented 9 years ago

Hooray! I suggest closing the issue then, and reopening it in case of any trouble ;)

jasisk commented 9 years ago

:+1:

Fishrock123 commented 9 years ago

Indeed. I'd gone ahead and said this was fixed as we knew in the 1.6.4 release. Closing, hopefully for good! :)

JCMais commented 9 years ago

Congratulations to everyone envolved. I was accompanying this issue, and was one of the best efforts to exterminate memory leaks that I've seen in my short history with open source projects. :clap:

capaj commented 9 years ago

wow they must love @indutny at google V8 team. Pushing features just so iojs can fix memory leak. Great stuff :ok_hand:

mscreenie commented 9 years ago

Congrats, your commitment to this problem and enthusiasm is outstanding and infectious. :+1:

indutny commented 9 years ago

Thanks guys!

interstellerS commented 9 years ago

This thread was exciting to watch as a node/io newbie , inspired by @indutny work . :+1:

devinus commented 9 years ago

@indutny What did you use for this screenshot? https://cloud.githubusercontent.com/assets/238531/6881510/064cb80e-d530-11e4-8a94-26ed428de317.png

gm112 commented 9 years ago

@devinus probably node-inspector or the blink developer tools

devinus commented 9 years ago

@gm112 My first guess was node-inspector, but it doesn't support io.js. I'm not sure how it could be Blink developer tools either.

gm112 commented 9 years ago

@devinus You can tell Blink Developer tools to connect to a remote debugging session, though I'm not 100% sure if this is just on v8 or a remote chrome instance. However Node-Inspector has limited support on iojs, some things work and some things don't. But I could be wrong :)

Olegas commented 9 years ago

@devinus I think this was a heap snapshot file generated by https://github.com/bnoordhuis/node-heapdump and opened in Chrome Developer Tools.