search

cypress-io / cypress

Fast, easy and reliable testing for anything that runs in a browser.
https://cypress.io
MIT License
46.97k stars 3.18k forks source link

Typescript Usage Results in Out of Memory Exception "JavaScript heap out of memory" #2316

Closed TLadd closed 5 years ago

TLadd commented 6 years ago

Current behavior:

When I try to use Typescript with Cypress, Cypress Helper's memory usage quickly scales up with the number of test files. The actual number of tests does not seem to be particularly important (i.e. it's fine to have 1 file with 100 tests in it, but 50 test files with 1 test each will break). After Cypress Helper's memory usage gets to a little of 2 GB in Activity Monitor, the tests grind to a halt and eventually result in an Out of Memory exception.

Desired behavior:

When using Typescript, the memory usage remains similar to how it behaves when not using typescript.

Steps to reproduce:

I have a repo setup that illustrates the issue: https://github.com/TLadd/cypress-ts-memory-leak/tree/master

It's a repo that only contains a basic cypress setup with typescript and @cypress/webpack-preprocessor. It contains 50 test files, each with one test each expect(1).to.eq(1);. For me, running cypress run consistently results in an out of memory exception trying to run the 37th test.

────────────────────────────────────────────────────────────────────────────────────────────────────

  Running: test42.ts...                                                                  (37 of 50)

<--- Last few GCs --->

[21977:0x7f94ab01b200]   151789 ms: Mark-sweep 2059.6 (2165.2) -> 2059.6 (2149.2) MB, 4886.1 / 0.0 ms  (+ 0.0 ms in 0 steps since start of marking, biggest step 0.0 ms, walltime since start of marking 4886 ms) last resort
[21977:0x7f94ab01b200]   156632 ms: Mark-sweep 2059.6 (2149.2) -> 2059.6 (2149.2) MB, 4842.6 / 0.0 ms  last resort

<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 0x2dfba37ad681 <JSObject>
    1: set [native collection.js:~247] [pc=0x928742bfdb4](this=0x28257ee942c9 <Map map = 0x44554988609>,p=0x1f93fc19e4f1 <String[13]: MSDescription>,x=0x1587c7882bf1 <SymbolObject map = 0x1f792a0791b1>)
    3: /* anonymous */(aka /* anonymous */) [/Users/thomasladd/github/cypress-ts-memory-leak/node_modules/typescript/lib/typescript.js:30020] [bytecode=0x3ff4e4d20bf9 offset=65](this=0x34f120f023...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
 1: node::Abort() [/Users/thomasladd/Library/Caches/Cypress/3.0.3/Cypress.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Libraries/libnode.dylib]
 2: node::FatalError(char const*, char const*) [/Users/thomasladd/Library/Caches/Cypress/3.0.3/Cypress.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Libraries/libnode.dylib]
 3: v8::internal::FatalProcessOutOfMemory(char const*) [/Users/thomasladd/Library/Caches/Cypress/3.0.3/Cypress.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Libraries/libnode.dylib]
 4: v8::internal::FatalProcessOutOfMemory(char const*) [/Users/thomasladd/Library/Caches/Cypress/3.0.3/Cypress.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Libraries/libnode.dylib]
 5: v8::internal::Factory::NewFixedArray(int, v8::internal::PretenureFlag) [/Users/thomasladd/Library/Caches/Cypress/3.0.3/Cypress.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Libraries/libnode.dylib]
 6: v8::internal::compiler::Node::Uses::empty() const [/Users/thomasladd/Library/Caches/Cypress/3.0.3/Cypress.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Libraries/libnode.dylib]
 7: v8::internal::RegisterConfiguration::AreAliases(v8::internal::MachineRepresentation, int, v8::internal::MachineRepresentation, int) const [/Users/thomasladd/Library/Caches/Cypress/3.0.3/Cypress.app/Contents/Frameworks/Electron Framework.framework/Versions/A/Libraries/libnode.dylib]
 8: 0x92873d043fd
 9: 0x928742bfdb4
10: 0x92873df78ea
11: 0x92873dbbe1c

Versions

Cypress: 3.0.3 OS: MAC High Sierra 10.13.4 Browser: Default Electron or Chrome

brian-mann commented 6 years ago

I was able to reproduce this, but I don't believe this is a Cypress specific issue.

The memory did not increase when I renamed all of the files to .js...

This problem only exhibits itself when using ts-loader.

Perhaps ts-loader is buffering every file in memory which is why you see the escalating memory usage. I did a cursory look around how we handle the pluginsFile, and it doesn't look like we're doing anything wrong. We keep the child process open for the entire duration of the run.

There may be some options in ts-loader that ask it to write its buffer to disk as opposed to memory, perhaps that would fix it.

There may also be webpack specific options you can pass to prevent this from happening as well.

Maybe someone on our team can look at this, but it seems this may not be an issue with Cypress.

TLadd commented 6 years ago

Thanks so much for looking into this. One thing that I've found that works is to pass the transpileOnly option to ts-loader. 

{
  loader: "ts-loader",
  options: {
    transpileOnly: true
  }
}

Whatever ts-loader is hanging onto across tests, it doesn't do it when in transpileOnly mode; the memory used by Cypress Helper goes up and down as each test completes and starts up, always staying in the range of a couple hundred MB.

bahmutov commented 6 years ago

That’s an excellent find!

Sent from my iPhone

On Aug 11, 2018, at 21:46, Thomas Ladd notifications@github.com wrote:

Thanks so much for looking into this. One thing that I've found that works is to pass the transpileOnly option to ts-loader.

{ loader: "ts-loader", options: { transpileOnly: true } } Whatever ts-loader is hanging onto across tests, it doesn't do it when in transpileOnly mode; the memory used by Cypress Helper goes up and down as each test completes and starts up, always staying in the range of a couple hundred MB.

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub, or mute the thread.

TLadd commented 6 years ago

Just an update on this, I tried updating to the latest ts-loader (v5.2.1), and even with the transpileOnly flag turned on, I end up running out of memory. Reverted back to 4.4.2 and not having issues.

masonmark commented 6 years ago

I was debugging why a seemingly-random set of our specs were failing with ERR_IPC_CHANNEL_CLOSED errors. Disabling parallelization (not passing --parallel) and running the tests sequentially made it obvious that we were running out of memory at the same spot, right after the 28th spec file, both on CI (Ubuntu 16.04) locally (macOS 10.14).

We are using ts-loader 4.3.0 with Cypress 3.1.0, and the workaround described by @TLadd above indeed worked for us.

Since we are using the "TypeScript with WebPack" example recipe as-is, this just meant adding the option transpileOnly to the webpackOptions structure inside our cypress/plugins/index.js file, like this:

      use: [{
        loader: 'ts-loader',
        options: {
          transpileOnly: true
        }
      }]
masonmark commented 6 years ago

I was debugging why a seemingly-random set of our specs were failing with ERR_IPC_CHANNEL_CLOSED errors. Disabling parallelization (not passing --parallel) and running the tests sequentially made it obvious that we were running out of memory at the same spot, right after the 28th spec file, both on CI (Ubuntu 16.04) locally (macOS 10.14).

We are using ts-loader 4.3.0 with Cypress 3.1.0, and the workaround described by @TLadd above indeed worked for us.

Since we are using the "TypeScript with WebPack" example recipe as-is, this just meant adding the option transpileOnly to the webpackOptions structure inside our cypress/plugins/index.js file, like this:

      use: [{
        loader: 'ts-loader',
        options: {
          transpileOnly: true
        }
      }]
masonmark commented 6 years ago

I was debugging why a seemingly-random set of our specs were failing with ERR_IPC_CHANNEL_CLOSED errors. Disabling parallelization (not passing --parallel) and running the tests sequentially showed that we were running out of memory at the same spot, right after the 28th spec file, both on CI (Ubuntu 16.04) locally (macOS 10.14).

We are using ts-loader 4.3.0 with Cypress 3.1.0, and the workaround described by @TLadd above indeed worked for us.

Since we are using the "TypeScript with WebPack" example recipe as-is, this just meant adding the option transpileOnly to the webpackOptions structure inside our cypress/plugins/index.js file, like this:

      use: [{
        loader: 'ts-loader',
        options: {
          transpileOnly: true
        }
      }]
jennifer-shehane commented 5 years ago

More specifically, ts-loader does have a section in their docs for optimizing faster builds here: https://github.com/TypeStrong/ts-loader#faster-builds

Olgagr commented 5 years ago

My setup is:

// package.json
"fork-ts-checker-webpack-plugin": "1.4.3",
"ts-loader": "5.3.0",
// webpack.config
const ForkTsCheckerWebpackPlugin = require('fork-ts-checker-webpack-plugin');

module.exports = {
  resolve: {
    extensions: ['.ts', '.js'],
  },
  module: {
    rules: [
      {
        test: /\.ts$/,
        exclude: [/node_modules/],
        use: [
          {
            loader: 'ts-loader',
            options: {
              transpileOnly: true,
              configFile: 'cypress/tsconfig.json',
            },
          },
        ],
      },
    ],
  },
  plugins: [
    new ForkTsCheckerWebpackPlugin({
      tsconfig: './cypress/tsconfig.json',
    }),
  ],
};

and the problem is gone as well. ForkTsCheckerWebpackPlugin is nice as it gives you the type checking in the separate process.

JoA-MoS commented 4 years ago

In my scenario when running my tests and running out of memory on spec 23 of 26 the testing continued to run even though no tests were actually being run and reported success even though the tests were not actually executed. Is this behavior correct? I would expect the remaining tests to fail. See screenshot below.

image

Is this by design or should this be a bug?

jennifer-shehane commented 4 years ago

@JoA-MoS All of the spec files listed at the top of the run under Specs should be listed completely at the end of the run - skipping files is not designed behavior from Cypress, unless you have some custom code doing this.

JoA-MoS commented 4 years ago

@jennifer-shehane - I was able to repro this in a demo repo (https://github.com/JoA-MoS/nx-example/tree/cypress-js-heap-oom). I think the issue is that once the TypeScript Preprocessor hits the out of memory it no longer processes the tests written in typescript. I believe 3.4.1 failed after finishing all the tests where 3.6.1 states all specs passed.

Here are the results of the demo repo

image

specs 23 and 24 contain the following

describe('demo-ui', () => {
  beforeEach(() => cy.visit('/'));

  it('I SHOULD NOT PASS', () => {
    // Function helper example, see `../support/app.po.ts` file
    cy.contains('THIS TEXT DOES NOT EXITS');
    expect(false).to.be.true;
  });
});

Spec 23 had the following logged to console.

 Running:  23.spec.ts                                                                    (24 of 25)
FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
 1: 00007FF679F407BF uv_open_osfhandle+479
 2: 00007FF678CB9786 node::Abort+22
 3: 00007FF678CB9DA3 node::Abort+1587
 4: 00007FF6772864F8 v8::RetainedObjectInfo::GetElementCount+808
 5: 00007FF677286493 v8::RetainedObjectInfo::GetElementCount+707
 6: 00007FF6772FF133 std::_Vector_alloc<std::_Vec_base_types<v8::CpuProfileDeoptInfo,std::allocator<v8::CpuProfileDeoptInfo> > >::_Make_iterator+255075
 7: 00007FF6772FE05B std::_Vector_alloc<std::_Vec_base_types<v8::CpuProfileDeoptInfo,std::allocator<v8::CpuProfileDeoptInfo> > >::_Make_iterator+250763
 8: 00007FF6772FB6BF std::_Vector_alloc<std::_Vec_base_types<v8::CpuProfileDeoptInfo,std::allocator<v8::CpuProfileDeoptInfo> > >::_Make_iterator+240111
 9: 00007FF677305409 std::_Vector_alloc<std::_Vec_base_types<v8::CpuProfileDeoptInfo,std::allocator<v8::CpuProfileDeoptInfo> > >::_Make_iterator+280377
10: 00007FF677305A8E std::_Vector_alloc<std::_Vec_base_types<v8::CpuProfileDeoptInfo,std::allocator<v8::CpuProfileDeoptInfo> > >::_Make_iterator+282046
11: 00007FF6776CF067 std::_Vector_alloc<std::_Vec_base_types<v8::CpuProfileDeoptInfo,std::allocator<v8::CpuProfileDeoptInfo> > >::_Make_iterator+4252567
12: 00007FF6778B20A9 std::_Vector_alloc<std::_Vec_base_types<v8::CpuProfileDeoptInfo,std::allocator<v8::CpuProfileDeoptInfo> > >::_Make_iterator+6231001
13: 00007FF677C0AFAF v8::WasmStreaming::Unpack+2677935

  0 passing (6ms)

  (Results)

  ┌────────────────────────────────────────────────────────────────────────────────────────────────┐
  │ Tests:        0                                                                                │
  │ Passing:      0                                                                                │
  │ Failing:      0                                                                                │
  │ Pending:      0                                                                                │
  │ Skipped:      0                                                                                │
  │ Screenshots:  0                                                                                │
  │ Video:        false                                                                            │
  │ Duration:     0 seconds                                                                        │
  │ Spec Ran:     23.spec.ts                                                                       │
  └────────────────────────────────────────────────────────────────────────────────────────────────┘

The following error was thrown by a plugin. We've stopped running your tests because a plugin crashed.

Running the demo repo:

git clone https://github.com/JoA-MoS/nx-example.git
git checkout cypress-js-heap-oom
npm install
ng e2e demo-ui-e2e --headless --prod

Related to: https://github.com/nrwl/nx/issues/2103