Out of memory with sitespeed version > 5.6.5

Pe4enie commented 6 years ago

Hi there,

was trying to update my configuration to latest major release (tried few different minor ones - 6.2.1, 6.1.3, 6.0.2) from the version I was using before - 5.6.5 and started constantly running into 'FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory'

After googling a little saw that people had similar issue only when they were running a lot of runs on thousands of urls. I currently get this error after 5 runs on 1 url consistently.

Here is a sample output:

Status: Downloaded newer image for sitespeedio/sitespeed.io:6.2.1
Google Chrome 63.0.3239.84
Mozilla Firefox 54.0.1
[2017-12-22 14:01:12] INFO: Versions OS: linux 4.4.0-93-generic nodejs: v8.9.1 sitespeed.io: 6.2.1 browsertime: 2.1.2 coach: 1.1.1
[2017-12-22 14:01:13] INFO: Starting chrome for analysing https://checkout.mytoys.de/checkout/registration 5 time(s)
[2017-12-22 14:01:13] INFO: Testing url https://checkout.mytoys.de/checkout/registration run 1
[2017-12-22 14:01:34] INFO: Testing url https://checkout.mytoys.de/checkout/registration run 2
[2017-12-22 14:01:56] INFO: Testing url https://checkout.mytoys.de/checkout/registration run 3
[2017-12-22 14:02:17] INFO: Testing url https://checkout.mytoys.de/checkout/registration run 4
[2017-12-22 14:02:37] INFO: Testing url https://checkout.mytoys.de/checkout/registration run 5
[2017-12-22 14:03:01] INFO: 52 requests, 572.17 kb, backEndTime: 1.49s (±306.05ms), firstPaint: 2.02s (±309.68ms), firstVisualChange: 2.08s (±306.70ms), DOMContentLoaded: 2.51s (±319.79ms), Load: 3.42s ($
FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory

<--- Last few GCs --->

[44:0x3795d00]   120148 ms: Mark-sweep 617.8 (665.7) -> 617.8 (665.7) MB, 491.7 / 0.0 ms  allocation failure GC in old space requested
[44:0x3795d00]   122558 ms: Mark-sweep 617.8 (665.7) -> 617.7 (625.2) MB, 2409.9 / 0.0 ms  last resort GC in old space requested
[44:0x3795d00]   123046 ms: Mark-sweep 617.7 (625.2) -> 617.7 (625.2) MB, 488.2 / 0.0 ms  last resort GC in old space requested

<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 0x13bd531a5ec1 <JSObject>
    1: getRedirectTarget [/usr/src/app/node_modules/pagexray/lib/util.js:~73] [pc=0xa439535b0b1](this=0x2a28b29c9a11 <Object map = 0x14bd9bc6f7b1>,url=0x105147824ca1 <String[48]: https://checkout.mytoys.$
    3: /* anonymous */(aka /* anonymous */) [/usr/src/app/node_modules/pagexr...

 1: node::Abort() [node]
 2: 0x12190dc [node]
 3: v8::Utils::ReportOOMFailure(char const*, bool) [node]
 4: v8::internal::V8::FatalProcessOutOfMemory(char const*, bool) [node]
 5: v8::internal::Factory::NewUninitializedFixedArray(int) [node]
 6: 0xe1bdb3 [node]
 7: v8::internal::Runtime_GrowArrayElements(int, v8::internal::Object**, v8::internal::Isolate*) [node]
 8: 0xa439518463d

And here is a script i'm running it with:

#!/bin/bash
DOCKER_CONTAINER=sitespeedio/sitespeed.io:6.2.1
DOCKER_SETUP="-e MAX_OLD_SPACE_SIZE=3072 --privileged --shm-size=1g --rm -v /home/sitespeed/sitespeed-config:/sitespeed.io -v /home/sitespeed/results:/result -v /etc/localtime:/etc/localtime:ro --name sitespeed-container"
THREEG="--network 3g"
CABLE="--network cable"
SITESPEED="/sitespeed.io"
CONFIG="--config $SITESPEED/default.json"
NO_LOGIN_URLS="$SITESPEED/no-login-urls.txt"
LOGIN_URLS="$SITESPEED/login-urls.txt"
MOBILE="-c 3g --mobile true"
PRELOAD_URL="--preURL https://checkout.mytoys.de/basketOverview"
LOGIN_SCRIPT="--preScript $SITESPEED/login-script.js"
NAMESPACE_PREFIX="--graphite.namespace sitespeed_io"
echo 'Start run ' >> /tmp/s.log 2>&1
docker network ls >> /tmp/s.log 2>&1

docker run $CABLE $DOCKER_SETUP $DOCKER_CONTAINER --browsertime.cacheClearRaw $CONFIG $NO_LOGIN_URLS >> /tmp/s.log 2>&1

where no-login-urls.txt is file which currently consists of 5 urls which do not require logged in state. As you can see I even tried increasing node old space memory pool. it didn't help.

Here is default.json I use (I changed real values to dummy values):

{
  "browsertime": {
    "connectivity": {
      "engine": "tc",
      "profile": "cable"
    },
    "iterations": 5,
    "browser": "chrome",
    "speedIndex": true
  },
  "graphite": {
    "host": "IP",
    "namespace": "sitespeed_io.desktop"
  },
  "resultBaseURL": "RESULT_BASE_URL",
  "gpsi": {
    "key": "GPSI_KEY"
  },
  "video": true,
  "gzipHAR": true,
  "html": {
    "fetchHARFiles": true
  },
  "s3": {
    "key": "KEY",
    "secret": "SECRET",
    "bucketname": "BUCKETNAME",
    "removeLocalResult": true
  },
  "firstParty": "checkout.mytoys.de"
}

I'm running this stuff on Ubuntu 16.04.3 LTS