samhatoum
commented
8 years ago From @Sanjo on October 7, 2015 19:14
Is the RSS column the memory usage? Is it in kB?
Note to me: Maybe Fiber leak (use Fiber pool everywhere instead)
Closed samhatoum closed 8 years ago
samhatoum
commented
8 years ago From @Sanjo on October 7, 2015 19:14
Is the RSS column the memory usage? Is it in kB?
Note to me: Maybe Fiber leak (use Fiber pool everywhere instead)
samhatoum
commented
8 years ago From @jamiecollinson on October 8, 2015 6:43
Yep - the RSS column is memory in kB. So the top node process is using over 900mb.
This is much higher than on our production server where the app takes 200 - 300mb depending on number of users.
On Wed, 7 Oct 2015 20:14 Jonas Aschenbrenner notifications@github.com wrote:
Is the RSS column the memory usage? Is it in kB?
Note to me: Maybe Fiber leak (use Fiber pool everywhere instead)
— Reply to this email directly or view it on GitHub https://github.com/xolvio/meteor-cucumber/issues/223#issuecomment-146300029 .
samhatoum
commented
8 years ago From @workflow on October 16, 2015 12:38
Same here with memory usage >4GB and only ~600 steps.
Cucumber: 0.14.11
Anything we can do to help isolate the culprit?
samhatoum
commented
8 years ago Do the # of steps affect the memory usage?
samhatoum
commented
8 years ago From @workflow on October 16, 2015 23:58
Ran a couple of tests (on Ubuntu 14.04, Cucumber 0.14.11) and came up with some hopefully interesting things:
General Memory usage after booting up and just before executing the first step seems very high. Especially given that the underlying meteor app itself uses < 300MB when running.
Total Mirror Memory Usage at the beginning of the first step (600 Steps): 2.0GB
Total Mirror Memory Usage at the beginning of the first step (60 Steps): 1.5GB
Total Mirror Memory Usage at the beginning of the first step (6 Steps): 1.3GBHere it get's interesting
That's the result of a test run with ~600 steps, all of them passing:
Full (600 steps, no failing ones):
Just b4 first step: 2.0GB
At finish: 2.4GB
SUCCESSFULLY COMPLETEDHere's the same run with 4 FAILING features (~60 steps or 10%) added.
Full Run (660 steps):
Just b4 first step: 2.2GB
Later: 2.6
Linearly increasing slowly with big jumps whenever a scenario fails
CRASHED AT 4.0GB RAM USAGE
FATAL ERROR: JS Allocation failed - process out of memoryThe step fail errors thrown are all of the kind
Error: element (#btnSettingsPopover) still not existing after 10000ms
at World.<anonymous> (features/Investor/discovering/companies/step_definitions/sort.js:17:17)In conclusio it would seem like failing steps (of the kind above) increase the memory usage significantly.
samhatoum
commented
8 years ago From @jamiecollinson on October 17, 2015 6:58
Anecdotal support for the failing tests use more memory theory - I raised this issue because our circleci builds were hitting the 4GB limit, and after spending some time this week refining our tests to be more robust (we had intermittent failures) our circleci builds just popped back to life and are working again.
I'll try and get some hard numbers after the weekend.
samhatoum
commented
8 years ago From @Sanjo on October 17, 2015 14:21
Maybe the screenshot blobs cause this significant increase of memory usage?
Can someone try it with those lines replaced (https://github.com/xolvio/meteor-cucumber/blob/master/src/mirror-server.js#L349-L351) with args.push('--screenshotsOnError=false'); (just clone this repo into the ./packages/ folder before).
samhatoum
commented
8 years ago I think that is the problem, I'm basically capturing the entire screenshot blob into the json report. It's probably better to save the screenshot to disk and reference it from the report.
I'll change this default behavior in the next release
samhatoum
commented
8 years ago From @workflow on October 18, 2015 20:59
Yep, that's it guys! (tested as @Sanjo suggested)
Yay :) @samhatoum - really looking forward to this one
What do you think about the base memory usage as reported above? Normal and expected? Or should I file a separate issue?
samhatoum
commented
8 years ago If the memory is general, then it makes sense since you have Java + chrome + Meteor + mongo etc. Would be good to get a breakdown
samhatoum
commented
8 years ago we've decommissioned xolvio:cucumber - please update to chimp - this shouldn't happen using it
mario-ortiz
commented
8 years ago I've tested this, it seems that I'm no longer getting the memory allocation error with v.0.30. But the JSON report also no longer have Screenshots, maybe there is a new parameter I should send?. This is the command I'm running
node_modules.bin\chimp --offline --screenshotsOnError=true --screenshotsPath=screenshots --captureAllStepScreenshots --attachScreenshotsToReport --saveScreenshots --format=pretty --jsonOutput=out\report.json --host=127.0.0.1 --port=4723 --platform=Android --name="Android Emulator" --tags=@mobile
It was working with version 0.22, but if I had more than 7 scenarios I was getting the memory allocation error. RIght now it's working, its just not attaching images anymore (blob or reference to file in disk).
mario-ortiz
commented
8 years ago There seems to be an error here: if (!stepResult.isSuccessful() && (booleanHelper.isTruthy(process.env['chimp.captureAllStepScreenshots']) || booleanHelper.isTruthy(process.env['chimp.screenshotsOnError']))) {
In the dist\lib\cucumberjs\hooks.js file (around line 45).
stepResult doesn't have a isSuccessful() function, so it never goes to take the screenshot. At least not in my environment. It breaks exactly in that line.
From @jamiecollinson on October 7, 2015 16:45
I recently upgraded Meteor to 1.2, and in the process upgraded
meteor-cucumberfrom 0.13.x -> 0.14.10. In the process I migrated to synchronous tests but didn't add anything new. Everything went pretty smoothly but I have noticed much higher memory consumption, to the extent that my CI builds on circleci are running out of memory and failing. The test suite is fairly large at ~1600 steps in all scenarios combined, but I imagine there must be people with much larger suites and I'd hope memory usage shouldn't scale with test steps anyway!I completely understand this may be velocity, meteor or something else other than meteor-cucumber, but didn't really know where to start looking and wondered if anyone else had experienced something similar? Maybe there's some obvious mistake I've made in translating the tests to synchronous style which could be the culprit?
The following is the memory usage as dumped from circleci when the tests hit the memory limit. Any help gratefully received:
Copied from original issue: xolvio/meteor-cucumber#223