nightwatchjs / nightwatch

Integrated end-to-end testing framework written in Node.js and using W3C Webdriver API. Developed at @browserstack
https://nightwatchjs.org
MIT License
11.79k stars 1.31k forks source link

JavaScript heap out of memory when writing out XML after tests finish #2771

Closed MicahLC closed 2 years ago

MicahLC commented 3 years ago

Describe the bug

When running on our CI server, Nightwatch sometimes crashes when writing out the test XML reports. I was able to capture the crash log.

Sample test

Not relevant

Verbose output

report.json

```txt { "header": { "reportVersion": 1, "event": "Allocation failed - JavaScript heap out of memory", "trigger": "FatalError", "filename": "report.20210614.152627.6232.0.001.json", "dumpEventTime": "2021-06-14T15:26:27Z", "dumpEventTimeStamp": "1623709587149", "processId": 6232, "cwd": "/home/ec2-user/workspace/redacted", "commandLine": [ "node", "/home/ec2-user/workspace/redacted/node_modules/.bin/nightwatch", "--skiptags", "redacted", "--env", "preprod", "--suiteRetries", "2" ], "nodejsVersion": "v12.16.1", "glibcVersionRuntime": "2.26", "glibcVersionCompiler": "2.17", "wordSize": 64, "arch": "x64", "platform": "linux", "componentVersions": { "node": "12.16.1", "v8": "7.8.279.23-node.31", "uv": "1.34.0", "zlib": "1.2.11", "brotli": "1.0.7", "ares": "1.15.0", "modules": "72", "nghttp2": "1.40.0", "napi": "5", "llhttp": "2.0.4", "http_parser": "2.9.3", "openssl": "1.1.1d", "cldr": "35.1", "icu": "64.2", "tz": "2019c", "unicode": "12.1" }, "release": { "name": "node", "lts": "Erbium", "headersUrl": "https://nodejs.org/download/release/v12.16.1/node-v12.16.1-headers.tar.gz", "sourceUrl": "https://nodejs.org/download/release/v12.16.1/node-v12.16.1.tar.gz" }, "osName": "Linux", "osRelease": "4.14.198-152.320.amzn2.x86_64", "osVersion": "#1 SMP Wed Sep 23 23:57:28 UTC 2020", "osMachine": "x86_64", "cpus": [ { "model": "AMD EPYC 7571", "speed": 2199, "user": 8845359300, "nice": 1200600, "sys": 722339600, "idle": 192980724700, "irq": 0 }, { "model": "AMD EPYC 7571", "speed": 2199, "user": 9172551300, "nice": 1215800, "sys": 704053700, "idle": 192658362300, "irq": 0 }, { "model": "AMD EPYC 7571", "speed": 2199, "user": 9139590000, "nice": 1175400, "sys": 737738100, "idle": 192694716100, "irq": 0 }, { "model": "AMD EPYC 7571", "speed": 2199, "user": 8141624700, "nice": 1070900, "sys": 674436000, "idle": 193732129000, "irq": 0 }, { "model": "AMD EPYC 7571", "speed": 2199, "user": 8606171300, "nice": 1068800, "sys": 713319500, "idle": 193248596600, "irq": 0 }, { "model": "AMD EPYC 7571", "speed": 2199, "user": 8251013900, "nice": 1043800, "sys": 694855900, "idle": 193628816400, "irq": 0 }, { "model": "AMD EPYC 7571", "speed": 2199, "user": 8137984600, "nice": 1141000, "sys": 696772800, "idle": 193740219300, "irq": 0 }, { "model": "AMD EPYC 7571", "speed": 2199, "user": 8881686800, "nice": 917300, "sys": 715328000, "idle": 192963757400, "irq": 0 } ], "networkInterfaces": [ { "name": "lo", "internal": true, "mac": "00:00:00:00:00:00", "address": "127.0.0.1", "netmask": "255.0.0.0", "family": "IPv4" }, { "name": "eth0", "internal": false, "mac": "06:33:da:f1:3b:e1", "address": "10.240.2.87", "netmask": "255.255.255.192", "family": "IPv4" }, { "name": "lo", "internal": true, "mac": "00:00:00:00:00:00", "address": "::1", "netmask": "ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff", "family": "IPv6", "scopeid": 0 }, { "name": "eth0", "internal": false, "mac": "06:33:da:f1:3b:e1", "address": "fe80::433:daff:fef1:3be1", "netmask": "ffff:ffff:ffff:ffff::", "family": "IPv6", "scopeid": 2 } ], "host": "ip-10-240-2-87.us-west-2.compute.internal" }, "javascriptStack": { "message": "No stack.", "stack": [ "Unavailable." ] }, "nativeStack": [ { "pc": "0x0000000000b278f5", "symbol": "report::TriggerNodeReport(v8::Isolate*, node::Environment*, char const*, char const*, std::string const&, v8::Local) [node]" }, { "pc": "0x00000000009f13ec", "symbol": "node::OnFatalError(char const*, char const*) [node]" }, { "pc": "0x0000000000b5da9e", "symbol": "v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [node]" }, { "pc": "0x0000000000b5de19", "symbol": "v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [node]" }, { "pc": "0x0000000000d0a765", "symbol": " [node]" }, { "pc": "0x0000000000d1b068", "symbol": "v8::internal::Heap::AllocateRawWithRetryOrFail(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [node]" }, { "pc": "0x0000000000ce8c8c", "symbol": "v8::internal::Factory::NewRawOneByteString(int, v8::internal::AllocationType) [node]" }, { "pc": "0x0000000001024a22", "symbol": "v8::internal::Runtime_AllocateSeqOneByteString(int, unsigned long*, v8::internal::Isolate*) [node]" }, { "pc": "0x00000000013a71b9", "symbol": " [node]" } ], "javascriptHeap": { "totalMemory": 2156683264, "totalCommittedMemory": 2155157520, "usedMemory": 2153451984, "availableMemory": 43226944, "memoryLimit": 2197815296, "heapSpaces": { "read_only_space": { "memorySize": 262144, "committedMemory": 33088, "capacity": 32808, "used": 32808, "available": 0 }, "new_space": { "memorySize": 1048576, "committedMemory": 1120, "capacity": 1047456, "used": 0, "available": 1047456 }, "old_space": { "memorySize": 62537728, "committedMemory": 62473080, "capacity": 62001936, "used": 62001936, "available": 0 }, "code_space": { "memorySize": 1216512, "committedMemory": 1084544, "capacity": 966528, "used": 966528, "available": 0 }, "map_space": { "memorySize": 2625536, "committedMemory": 2572920, "capacity": 1968640, "used": 1968640, "available": 0 }, "large_object_space": { "memorySize": 2088943616, "committedMemory": 2088943616, "capacity": 2088479288, "used": 2088479288, "available": 0 }, "code_large_object_space": { "memorySize": 49152, "committedMemory": 49152, "capacity": 2784, "used": 2784, "available": 0 }, "new_large_object_space": { "memorySize": 0, "committedMemory": 0, "capacity": 1047456, "used": 0, "available": 1047456 } } }, "resourceUsage": { "userCpuSeconds": 49.0928, "kernelCpuSeconds": 5.79491, "cpuConsumptionPercent": 2.10459, "maxRss": 2215571456, "pageFaults": { "IORequired": 0, "IONotRequired": 1649584 }, "fsActivity": { "reads": 0, "writes": 13664 } }, "uvthreadResourceUsage": { "userCpuSeconds": 33.6827, "kernelCpuSeconds": 4.47213, "cpuConsumptionPercent": 1.46299, "fsActivity": { "reads": 0, "writes": 4536 } }, "libuv": [ ], "environmentVariables": "REDACTED", "userLimits": { "core_file_size_blocks": { "soft": 0, "hard": "unlimited" }, "data_seg_size_kbytes": { "soft": "unlimited", "hard": "unlimited" }, "file_size_blocks": { "soft": "unlimited", "hard": "unlimited" }, "max_locked_memory_bytes": { "soft": 65536, "hard": 65536 }, "max_memory_size_kbytes": { "soft": "unlimited", "hard": "unlimited" }, "open_files": { "soft": 4096, "hard": 4096 }, "stack_size_bytes": { "soft": 8388608, "hard": "unlimited" }, "cpu_time_seconds": { "soft": "unlimited", "hard": "unlimited" }, "max_user_processes": { "soft": 127455, "hard": 127455 }, "virtual_memory_kbytes": { "soft": "unlimited", "hard": "unlimited" } }, "sharedObjects": [ "linux-vdso.so.1", "/lib64/libdl.so.2", "/lib64/libstdc++.so.6", "/lib64/libm.so.6", "/lib64/libgcc_s.so.1", "/lib64/libpthread.so.0", "/lib64/libc.so.6", "/lib64/ld-linux-x86-64.so.2" ] } ```

Configuration

nightwatch.json

```js { src_folders: ['e2e/test'], output_folder: './e2e/reports', globals_path: './e2e/globals.js', custom_commands_path: './e2e/custom/browserCommands', custom_assertions_path: './e2e/custom/assertions', page_objects_path: './e2e/pages', persist_globals: false, end_session_on_fail: false, skip_testcases_on_fail: true, uncaughtErrorPublishReport: true, detailed_output: false, output_timestamp: true, live_output: false, disable_colors: true } ```

Your Environment

Executable Version
nightwatch --version v1.6.3
npm --version v6.13.4
node --version v12.16.1
Browser driver Version
NAME VERSION
chromedriver 90
OS Version
NAME VERSION
Amazon Linux 2
beatfactor commented 3 years ago

Thanks for reporting this, we'll look into it.

gravityvi commented 3 years ago

Hey @MicahLC, can you provide some more details regarding this issue like what runner are you using (Nightwatch default runner, mocha runner), the reporter, and CI config which would help me to reproduce this issue.

MicahLC commented 3 years ago

We use the Nightwatch default runner. We are using an internally-maintained variant of this reporter with some visual modifications, but little to no changes to logic.

By CI config, do you mean some info about the environment where this is running? This is running inside a docker node inside a Jenkins pipeline on an AWS EC2 instance. We spin up docker images running our app, docker images to run the selenium hub and the selenium chrome instances, and docker images to run the tests using docker-compose. The crash occurs on the docker image running the tests (obv :) ), but after the tests have completed and the HTML report has been written out.

stale[bot] commented 2 years ago

This issue has been automatically marked as stale because it has not had any recent activity. If possible, please retry using the latest Nightwatch version and update the issue with any relevant details. If no further activity occurs, it will be closed. Thank you for your contribution.