Wndows CPU & Memory management

[halfabench]

Hi, I'm using hw on windows 7 and It uses a lot of CPU and memory.

For my current js project , it takes up to 65% of my CPU (Intel i5) + 400MO RAM when making a new compilation and then it takes about 15sec to come back to 0 CPU + 110MO RAM.

I will try to make more tests so I can give you more information. Anyway thanks for this project. It's a great initiative! Than you :)

[lucamezzalira]

thanks for report, I've downloaded yesterday night a Windows 8.1 iso (and an Ubuntu one) and tonight I'm going to find a Windows 7 one so I'll be able to perform some test.

[halfabench]

Great!

[lucamezzalira]

Sorry but tonight I worked on Linux to see if there were the same issue. I was thinking, did you try to run in server mode instead of with the default compiler? Maybe that should help to optimise the performance. You can find how to setup your project in this way in the README file, let me know if you'll have the same issue

[lucamezzalira]

I don't know if running on a VM could change drastically but with FlashDevelop and haxe watchify I didn't see terrible spikes of memory or CPU usage in a simple project. I'll investigate further on a Win 7 but for now Win 8.1 looks fine. BTW try to set the completion server should be definitely better!

[halfabench]

OK, using server is indeed way faster and takes 0% CPU + 50M ! So that's pretty good! But now I get some weird macro errors I don't get when not using --compiler server. But I don't thinks it's related to HW.

If I read the server doc on haxe.org: Macros: The compilation server can have some side effects. So it seems the jquery lib I use, does not work well with the compilation server...

[lucamezzalira]

yeah I know about that issues, unfortunately macros aren't well supported on completion server! Probably in for that you should ask to the haxe foundation guys if there is any plan so solve this issue and potentially when

BTW sooner than later I'm going to add the implementation of the livereload so maybe with that one, refactoring the code and killing the processes spawned you'll have better performance as well without the completion server.

[halfabench]

ok I'll do that! Thanks!

[halfabench]

Ok upgrading the jQueryExtern lib did the trick!

[halfabench]

Ok. Been working yesterday and this morning with HW + compilation server. I have a few questions: 1 - memory usage is increasing over time. Is it related to file caching ? A leak ? 2 - when raising an error, HW seems to be waiting longer before taking new changes into account. Is it the wanted behaviour ? And it does take more CPU (50%). 3 - saving multiple times before the current compilation is done makes the memory consumption bigger. For instance making 3 saves in a row, leading to 3 compilations processes, increase my memory usage by 20M on the current project. 4 - linked to these behaviours : the fact that compilation eventually takes more time after a few hours.

So after a few hours, I shut down HW to get a new "clean" context. It just seems the compilation server allows for a better start, but eventually ends up the same way that HW without it. Hope this helps ! :)

[lucamezzalira]

Hi,

I'm working in these days on a new version where these issues should be addressed in a better way for sure.

I hope next week to be able to release it!

For now thanks so much for your feedback and for using HW :D I keep you updated on the next releases!

[halfabench]

Great! Thanks! Just FYI, HW went just berserk with memory (400M) and CPU (50%) for about 5 minutes. It seems to be related to me wanting to compile many times (=saving), each compilation leading to an error and each compilation asked before the previous one was complete. So It seems like a mix of the behaviours I described.

[lucamezzalira]

yep, I think I know how to solve it but thanks for your benchmark, I really appreciate it :+1:

[halfabench]

you're welcome :) Good luck!

[halfabench]

Did a long run yesterday, ended up with: "FATAL ERROR: CALL_AND_RETRY_2 Allocation failed - process out of memory" (seems to be nodejs + v8 memory problem )

I have not updated haxe-watchify ( the npm version has not changed yet ) but I thought the message could give you some clue about some memory leak. Do you want me to post a new issue or do I wait for the next version to come and then we'll see?

[lucamezzalira]

Please wait the next one I'm invastigating on that in these days

[halfabench]

ok.

[lucamezzalira]

I've to ask you if you can check out the master branch and try to run haxe-watchify from there instead using the global one to understand if something as changed on windows performance. I see a better management of CPU on windows also on small projects so maybe this thing could help:

1. checkout the project on master branch
2. from the folder you've just check out run:

npm install

1. from your folder project run in your CLI:

node --max-old-space-size=4000 path/to/your/haxe-watchify-folder/src/haxe-watchify.js --program openfl --platforms flash,html5

let me know if you have any problems thanks in advance

[halfabench]

ok done. The node process takes no cpu, memory usage has been very stable( 55000-56000K ) since I started the process (5 minutes ago). Now is it supposed to output anything or is just to check if things remain stable?

[lucamezzalira]

good, I was just checking things remain stable, so this week I'm going to finish the next release with livereload feature inside, I hope next week to finalise the documentation and then you'll enjoy haxe-watchify :+1:

[halfabench]

Great! Thanks!

[lucamezzalira]

please check the new release 1.2.0

[halfabench]

Ok. Did a few tests with many saves with / without errors in a very short amount of time. It seems to work better: after a failed compilation, it is not stuck anymore, memory consumption seems good, and all the builds occurred very fast. So it seems to handle better.

It needs some more testing but I am on another project (using scala) these days so I won't use haxe much. So I will try report to you when I make a long Haxe session.

Anyway thanks for the good work!

[sruloart]

I think this can also be considered as fixed, AFAIK.

[lucamezzalira]

I'd leave it open till @halfabench ends his tests

[halfabench]

Had to work all day yesterday on haxe so I can confirm everything's working well :+1: Bravo!

[lucamezzalira]

nice! :D