WMIC.exe not cleaned up on Windows

[rexpie]

When PM2 stopped working, I checked process explorer and see a long list of open WMIC.exe processes. PM2 can work with a few lingering WMIC.exe but as time goes it will stop responding altogether.

The OS I am using does not have the WMIC.exe so I copied from somewhere else and put the program in the working directory of the project. Before that PM2 is not working at all. I checked the error log and npm page of pidusage to find out that this module depends on WMIC.exe.

I wonder if the path has something do to with those lingering programs?

Many thanks in advance

[soyuka]

May I ask what is you windows version?

I wonder if the path has something do to with those lingering programs?

Yes it might, because we have to avoid spawning cmd.exe windows, we have to launch wmic.exe in a detached mode. That said, if your own version of wmic is failing (eg not exiting) or loops in the background, pidusage has no clue that it's not responding.

Can you try launching the command manually? If you have any idea on how to improve windows process usage/memory detection, I'm all ears!

[rexpie]

Hi, I am using win7 x64 sp1. But I think it might be pirated. I have tried to lauch wmic path Win32_PerfFormattedData_PerfProc_Process WHERE IDProcess=xxxx get PercentProcessorTime, WorkingSet and it gets stuck. But it returns immediately if I use wmic PROCESS 6968 get workingsetsize

[soyuka]

Hmm, might be that I had a similar problem but I found no fix to my issue... Check https://github.com/soyuka/pidusage/pull/16.

Thing is, I have no way to check that wmic path Win32_PerfFormattedData_PerfProc_Process WHERE IDProcess works, nor can I send both commands to see which one works on the computer, or it'll eat a lot more ressources that it currently does.

I've looked at adding a windows binary in C#, but I don't know if it'll be as portable as wmic. Wmic should work on most windows versions > XP.

Let me know if I can do anything.

[rexpie]

That's okay, this is bound to happen when we have to rely on OS specific tools. I noticed that PM2 module uses this module to provide CPU and memory info, so I commented out that bit since that info is not crucial to me. Now PM2 does not get stuck any more. Thanks again for such prompt and detailed feedback. I am closing this issue for the moment.

[3eggert]

Can you tell what you commented out? I have the same issue...

[soyuka]

@3eggert

Can you try launching the command manually? If you have any idea on how to improve windows process usage/memory detection, I'm all ears!

[3eggert]

I'm afraid windows is not my weapon of choice, so I have no improvement ideas... but, sure I can call it manually, if you tell me how to figure out the process id needed in the line. Btw. til now I was not able to comment out lines to make pm2 usable again, any idea how to disable wmic.exe usage?

[soyuka]

Sorry I was just quoting a previous reply of mine. Neither is windows my weapon of choice ;).

Process id can be found in the windows process manager or in the pm2 list or process.pid global variable in node.

If you want to prevent wmic from running you can add this line here:

      return done(null, {cpu : 0, memory : 0})

[3eggert]

Thanks for that line, seems to work now. I also run the command manually:

C:\Users\bbuser>wmic path Win32_PerfFormattedData_PerfProc_Process WHERE IDProcess=5424 get PercentProcessorTime, Workin
gSet
PercentProcessorTime  WorkingSet
0                     21262336

C:\Users\bbuser>

I took quite long, at least 3sec I would say and consumed 2 of 4 cores with 80%. Seems it should not be called in a short loop, other wise it starts many even before the first had finished. Probably that was at least one of the problems in my case, pm2 started more and more of it until the machine cried for a powercycle after a minute or so...

[soyuka]

Really interesting thx, I'll check that pm2 waits for a response before issuing a new stat! Thx! Le mer. 22 juin 2016 à 18:54, Thorsten Eggert notifications@github.com a écrit :

Thanks for that line, seems to work now. I also run the command manually:

C:\Users\bbuser>wmic path Win32_PerfFormattedData_PerfProc_Process WHERE IDProcess=5424 get PercentProcessorTime, Workin gSet PercentProcessorTime WorkingSet 0 21262336

C:\Users\bbuser>

I took quite long, at least 3sec I would say and consumed 2 of 4 cores with 80%. Seems it should not be called in a short loop, other wise it starts many even before the first had finished. Probably that was at least one of the problems in my case, pm2 started more and more of it until the machine cried for a powercycle after a minute or so...

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/soyuka/pidusage/issues/19#issuecomment-227806944, or mute the thread https://github.com/notifications/unsubscribe/ABQr87oDw1cgMcCnamGgQIibI_8vnVa0ks5qOWisgaJpZM4IGFDD .

[3eggert]

Sounds like a plan, perhaps you should even wait a sec or more (perhaps make it cfg able) after. As I said its not a lightweight call, and a update every sec should be good enough.

[soyuka]

It's waiting for the callback before issuing a new timeout: https://github.com/Unitech/pm2/blob/30239e142928ae04fa19733e186df4c9288f3949/lib/CLI.js#L1575. At least with pm2 monit.

[erenayg]

Hi,

I believe I experience the same issue, which causes many WMIC.exe instance to be spanned, eating all CPU on the host.

I see the issue is marked as closed but wasn't able to see the solution.

In my case the host is Window Server 2012 R2. PM2 version: 2.2.3

The problem is, once pm2 is up, almost every second a new WMIC.exe is spanned. For some time, these are cleaned up (exe terminates). However, after some time (i.e. some hours), WMIC.exe start piling up... eventually they end up consuming CPU 100%. Server becomes almost dead.

Windows Event viewer shows more than 1000 errors for 10 minutes of WMI-Activity, all with the same error type. https://www.screencast.com/t/8m1LaZe0u4

As a result, I had to disable keymetrics connection (pm2 interact delete), to avoid WMIC.exe usage.

I believe this issue blocks monitoring with pm2 on Windows hosts...

Regards.

[soyuka]

@erenayg wow this sucks!

I'll see if I can find a solution for you to keep monitoring (by reducing the number of calls to pidusage).

I've tried many many hacks and I believe that the current solution on pidusage can't get better (if I stick with wmic). What I don't understand is why wmic is so process consuming...

nb error code: 0x80041033

Thanks for your report!

[yoyo497]

It seems windows xp can't spawn the wmic process correctly which result in an unclosed wmic command prompt and will only return on.close results by killing it yourself. Adding a space after 'wmic' in the spawn code fixed the problem for me.

lib/stats.js spawn('wmic ', args.split(' '), {detached: true})

[soyuka]

Wow okay, do you know why? Have you access to another windows version to try if this hack also works there?

Thanks!!

[yoyo497]

For some reason the wmic didn't hang on our windows 7 systems using pm2. I have added the space to the stats.js file to see if anything brakes but it seems to work ok on windows 7 as well.

[soyuka]

Okay thank you very much, I'll patch this asap!

[yoyo497]

Solves a major problem for us, thx!

[soyuka]

I had to revert, unit tests are failing on appveyor :|. I'll see if I can get more informations

[soyuka]

Appveyor is apparently running on server R2, @yoyo497 I tried to add your fix only on older versions of windows, would you be able to run the test suite on windows xp?

git clone https://github.com/soyuka/pidusage
cd pidusage
npm install
npm test

[yoyo497]

Windows xp machine fails:

1> Pid usage should get pid usage:

Uncaught AssertionError: expected [Error: Fri Apr 28 2017 08:02:13 GMT+0100 Wmic errored, please open an issue on https://github.com/soyuka/pidusage with this message. Command was "wmic PROCESS 3708 get workingsetsize,usermodetime,kernelmodetime" System informations:

• release: 5.1.2600
• type: Windows_NT Wmic reported no errors . Stdout was empty] to be null at C:\\node_modules\pidusage\test\test.js:47:24 at ChildProcess. <C:\\node_modules\pidusage\lib\stats.js:155:16> at maybeClose <internal/child_process.js:829:16> at Socket. <internal/child_process.js:319:11> at Pipe._onclose

Windows 7 machine all test passed.

[soyuka]

Hmm, so the space doesn't change anything but instead is breaking the command... I really wonder how you did make this work. My change goes as follow:

    var wmic = spawn(releaseMajor < 6 ? 'wmic ' : 'wmic', args.split(' '), {detached: true})

So, it should apply the same fix as you...

[yoyo497]

Yes i think your rigth. Im out of office till monday. I will do some more research why wmic wont close properly on windows xp.

[soyuka]

This would be greatly appreciated! Let me know if you need any information, I did a lot of digging on WMIC but the windows documentation is a mess!

[yoyo497]

Problem fixed on xp adding this line:

wmic.stdin.end();

All test pass now :)

[soyuka]

Would you mind opening a pull request with your fixes? Thanks!

[soyuka]

thanks to @yoyo497 this should be resolved, feel free to ping if not.