PM2 using too much RAM and CPU

[frontierpsycho]

Now, I know this issue is potentially very case-specific, but I'd like to get your thoughts on it, and also let you know of it.

I have written a nodejs service that is supposed to run forever, listen to a queue, and act upon the messages. The actions taken include appending to a file and writing to a database. The entire thing is asynchronous.

Since I was going to run this on a dual-core machine, and potentially even more cores in the future, I wanted to run more than one instance. I wanted to use PM2's clustering, because it seemed very convenient, and PM2 in general seems to come with lots of goodies which I liked (the web interface, clustering, log aggregation, etc). I also considered forever, which has only the basic of PM2's features.

However, after running two instances of the service for a day with each of forever and PM2, the results are:

• forever: ~250mb RAM, ~20% CPU usage
• PM2: ~1,250mb RAM, ~100% CPU usage

The machines are virtual, and exactly the same. I tried with both node 0.10 and 0.12, and the results were the same.

Because under PM2, the instances are draining the machine's resources, the service performs atrociously (sometimes even consuming 10 times more slowly).

My question is mainly, are you aware of any performance weaknesses of PM2? Are there cases where PM2 is not the correct solution? Do you think you could fix/optimize the problem? It seems likely that this is a memory leak (probably not in my code, as forever seems to run it quite well).

I would have really liked to use PM2 for this.

[jshkurti]

Hello, can you try it with npm i -g git://github.com/Unitech/PM2#development version please ? I removed a feature which I suspect to be the cause of the memory leak.

[jshkurti]

Also, which version of PM2 are you using ? Are your apps in fork mode or cluster mode ?

[frontierpsycho]

My PM2 reports version 0.12.15, and I've been using cluster mode.

I'm going to try to use the development version as soon as possible, although that might not be very very soon. Thanks for the reply, though.

[Unitech]

When you report such apparently critical things please attach a screenshot of pm2 resources usage with htop.

We use PM2 under heavy load on 6 differents servers, with cluster mode with all keymetrics features and we never seen that.

[frontierpsycho]

Unfortunately, I was working on vagrant machines which I've now destroyed.

To be clear, I'm not suggesting PM2 is always leaking memory and that it's useless, but I've seen a couple of mentions of this online, so I believe under certain circumstances this might be true. Hopefully the bugfix jshkurti mentioned will fix those.

[Unitech]

Ok we keep this issue open for the week if there is someone else experiencing that issue.

Which modules were you using? (queue etc)

[frontierpsycho]

No modules.

[Unitech]

Could you send us a gist of your app so we can try to reproduce the issue?

Thanks

[frontierpsycho]

Unfortunately I can't, I wrote it at my work and it's not open-source. Plus you probably can't reproduce it, as it isn't standalone, it interacts with several systems in our architecture, a queue, the DB, etc. If I have time, I can try to present you with a pared down version. I'm more likely to just try the dev version and see how that works, though, to be honest.

[frontierpsycho]

Alright, so I tried the development version, left it to work for a day (although without much traffic), and it seems to work okay! I got a memory usage of ~250mb, and reasonable CPU usage.

So, what is the fix? Is it going to be released soon?

[Unitech]

Thanks for trying it!

We are still deep inspecting the issue, would it be possible to have a sample of your code that uses the http or the https module?

[elijahchancey]

+1.

I'm experiencing a similar issue. I've installed PM2 and have not yet configured any apps (haven't yet configured fork vs cluster). Using NVM to install node v0.12.4 and then using npm the latest pm2. Here's an htop screenshot: https://www.dropbox.com/s/j0v4p1d9kui0pg2/Screenshot%202015-06-07%2022.19.24.png?dl=0 Ubuntu 14.04.2 PM2 0.12.16

Let me know what other information I can provide.

[Unitech]

Just spawned a Ubuntu 14.04 on AWS, with 2 cores and 4go, with Node.js 0.12.4:

What did you do before to get such a high CPU/Memory usage?

[Unitech]

Bug reproduced - inspecting

[frontierpsycho]

BTW, I don't really use the http or https modules directly, I use this:

https://github.com/almost/sqs-readable-stream

and this:

https://github.com/felixge/node-mysql/

[askhogan]

@frontierpsycho @elijahchancey

Add this to your project and retry

require('pmx').init({
    http: false, 
    errors: false,
    custom_probes: false,
    network: false, 
    ports: false  
});

My memory issue was solved after setting http to false. By default http is on.

https://github.com/keymetrics/pmx/issues/23

[frontierpsycho]

Wait, I wasn't using keymetrics. Does PM2 by default send data to keymetrics?

Also, I won't be able to try it, as I opted for a different solution right now (forever). Moreover, since the development version fixed it for me, I guess after that's released, it will no longer be an issue.

[askhogan]

If you weren't using keymetrics, then it shouldn't send data. If you setup keymetrics it does.

Glad to hear the dev version solved it for you. Will give that a shot.

[jaredhirsch]

@Unitech I see you landed some commits related to this issue. Have you verified that it's resolved?

[ajschlosser]

I am having this issue using PM2 0.14.7 on Ubuntu 14.04 with Node.js 4.2.1.

We have four cores; PM2 always uses >100% of one of them, and ~50% of another. It also uses >90% of our available RAM.

We have tried various combinations of Node.js and PM2 versions to no avail.

[qspencer]

Same problem - Ubuntu 14.04, 8 GB, 4 CPUs (on AWS EC2 if that matters), Node.js 4.1.2, PM2 0.15.7

[soyuka]

Actually it also uses a lot of cpu, can we have last 100 lines of ~/.pm2/pm2.log? The json configuration might be interesting too, thx.

[qspencer]

Not using json configuration (unless there's a default). Log attached.

On Thu, Oct 29, 2015 at 2:55 PM, Antoine Bluchet notifications@github.com wrote:

Actually it also uses a lot of cpu, can we have last 100 lines of ~/.pm2/pm2.log? The json configuration might be interesting too, thx.

— Reply to this email directly or view it on GitHub https://github.com/Unitech/pm2/issues/1314#issuecomment-152286442.

[soyuka]

Do you use options (like watch)?

[qspencer]

no, I don't. I was trying to call PM2 restart Githubissues.

Githubissues is a development platform for aggregating issues.