search

vsivsi / meteor-job-collection

A persistent and reactive job queue for Meteor, supporting distributed workers that can run anywhere.
https://atmospherejs.com/vsivsi/job-collection
Other
388 stars 68 forks source link

Memory Leaks? #201

Closed pozylon closed 6 years ago

pozylon commented 7 years ago

Hi there

I guess meteor job collection leaks memory over time, but I'm wondering why there is no issue posted here yet. Nobody else?

Look at the screenshot here: the only difference is crossiety-one doesn't startJobServer() whereas crossiety-two does.

bildschirmfoto 2016-11-04 um 13 25 07

What I've found out so far:

  1. I tried hard to track down issues with my own code and found out that even if you use an empty function as job, the leak still happens.
  2. I could not yet find out if it's just the gc waiting for the old-memory to become 1400mb or if it is a real leak.

Things to try:

  1. Expose GC http://stackoverflow.com/questions/20473183/expose-garbage-collector-in-a-node-js-app-with-strongloop-support and call inside a gc job.
  2. Examine meteor-job-collection code with tests that simulate a long-running job server and snapshot memory with node inspector.
vsivsi commented 7 years ago

As you noted, I haven't received any reports of memory leakage in job collection until now. But I would clearly need much more information than you provide to be able to reproduce what you are seeing and try to help track down whether the perceived leak is real or a consequence of delayed GC. If it is real, then of course it still wouldn't be immediately clear that job-collection itself is the culprit.

If you'd like to proceed, please create and share a github repo and instructions to reproduce the results in the screenshot above, and I'll be happy to take a look.

Chugot commented 7 years ago

We run several Meteor programs that utilize JobCollection 24/7 on dozens of windows clients and we have not noticed any lasting performance issues or memory leaks, just my two cents.