Open vespertilian opened 1 year ago
This is extremely critical, it's literally unusable now. Any temporary workarounds, please?
I've tried to run your reproducing repo with chrome inspector and do not see any memory leaks. Yes, jest reports memory growth for each file BUT it is normal behaviour because of the Garbage Collector didn't triggered yet. If i manually trigger GC then memory footprint becomes ~160MB.
The slowness that i see happens because of for each spec file Jest executes setup-jest.js
file that does this:
require('zone.js/bundles/zone-testing-bundle.umd');
const { getTestBed } = require('@angular/core/testing');
const {
BrowserDynamicTestingModule,
platformBrowserDynamicTesting,
} = require('@angular/platform-browser-dynamic/testing');
const testEnvironmentOptions = globalThis.ngJest?.testEnvironmentOptions ?? Object.create(null);
getTestBed().initTestEnvironment(BrowserDynamicTestingModule, platformBrowserDynamicTesting(), testEnvironmentOptions);
As we can see that for each simple spec file:
describe('AppComponent', () => {
it('dummy test', () => {
expect(1).toEqual(1);
});
});
we bootstrap the Angulars testing module.
Without extra bootstrap logic tests pass in 13 seconds instead of 40 seconds in my laptop.
Just to confirm my assumptions regarding memory leak we can launch nodejs instance with the hard limit by available memory to force V8 GC to run much frequently. Lets limit heap size to 300MB:
jest-preset-angular-mem-usage % node --max-old-space-size=300 node_modules/.bin/jest --runInBand --logHeapUsage --detectOpenHandles
PASS src/app/app.component.94.spec.ts (97 MB heap size)
PASS src/app/app.component.26.spec.ts (120 MB heap size)
PASS src/app/app.component.73.spec.ts (153 MB heap size)
PASS src/app/app.component.76.spec.ts (157 MB heap size)
PASS src/app/app.component.1.spec.ts (192 MB heap size)
PASS src/app/app.component.11.spec.ts (194 MB heap size)
PASS src/app/app.component.96.spec.ts (204 MB heap size)
PASS src/app/app.component.100.spec.ts (237 MB heap size)
PASS src/app/app.component.78.spec.ts (242 MB heap size)
PASS src/app/app.component.79.spec.ts (160 MB heap size)
PASS src/app/app.component.63.spec.ts (159 MB heap size)
PASS src/app/app.component.5.spec.ts (197 MB heap size)
PASS src/app/app.component.4.spec.ts (186 MB heap size)
PASS src/app/app.component.77.spec.ts (196 MB heap size)
PASS src/app/app.component.23.spec.ts (228 MB heap size)
PASS src/app/app.component.39.spec.ts (234 MB heap size)
PASS src/app/app.component.24.spec.ts (238 MB heap size)
PASS src/app/app.component.87.spec.ts (152 MB heap size)
PASS src/app/app.component.57.spec.ts (154 MB heap size)
PASS src/app/app.component.58.spec.ts (185 MB heap size)
PASS src/app/app.component.55.spec.ts (168 MB heap size)
PASS src/app/app.component.64.spec.ts (202 MB heap size)
PASS src/app/app.component.37.spec.ts (207 MB heap size)
PASS src/app/app.component.56.spec.ts (230 MB heap size)
PASS src/app/app.component.75.spec.ts (239 MB heap size)
PASS src/app/app.component.13.spec.ts (244 MB heap size)
PASS src/app/app.component.97.spec.ts (155 MB heap size)
PASS src/app/app.component.95.spec.ts (162 MB heap size)
PASS src/app/app.component.48.spec.ts (195 MB heap size)
PASS src/app/app.component.15.spec.ts (171 MB heap size)
PASS src/app/app.component.16.spec.ts (203 MB heap size)
PASS src/app/app.component.10.spec.ts (209 MB heap size)
PASS src/app/app.component.80.spec.ts (230 MB heap size)
PASS src/app/app.component.spec.ts (211 MB heap size)
PASS src/app/app.component.9.spec.ts (219 MB heap size)
PASS src/app/app.component.74.spec.ts (252 MB heap size)
PASS src/app/app.component.61.spec.ts (243 MB heap size)
PASS src/app/app.component.68.spec.ts (144 MB heap size)
PASS src/app/app.component.21.spec.ts (145 MB heap size)
PASS src/app/app.component.41.spec.ts (178 MB heap size)
PASS src/app/app.component.52.spec.ts (172 MB heap size)
PASS src/app/app.component.31.spec.ts (207 MB heap size)
PASS src/app/app.component.53.spec.ts (212 MB heap size)
PASS src/app/app.component.42.spec.ts (234 MB heap size)
PASS src/app/app.component.65.spec.ts (215 MB heap size)
PASS src/app/app.component.27.spec.ts (223 MB heap size)
PASS src/app/app.component.46.spec.ts (256 MB heap size)
PASS src/app/app.component.40.spec.ts (245 MB heap size)
PASS src/app/app.component.34.spec.ts (146 MB heap size)
PASS src/app/app.component.3.spec.ts (155 MB heap size)
PASS src/app/app.component.71.spec.ts (180 MB heap size)
PASS src/app/app.component.43.spec.ts (181 MB heap size)
PASS src/app/app.component.92.spec.ts (192 MB heap size)
PASS src/app/app.component.50.spec.ts (223 MB heap size)
PASS src/app/app.component.90.spec.ts (230 MB heap size)
PASS src/app/app.component.18.spec.ts (253 MB heap size)
PASS src/app/app.component.14.spec.ts (221 MB heap size)
PASS src/app/app.component.12.spec.ts (201 MB heap size)
PASS src/app/app.component.81.spec.ts (208 MB heap size)
PASS src/app/app.component.59.spec.ts (239 MB heap size)
PASS src/app/app.component.49.spec.ts (200 MB heap size)
PASS src/app/app.component.88.spec.ts (193 MB heap size)
PASS src/app/app.component.62.spec.ts (224 MB heap size)
PASS src/app/app.component.66.spec.ts (223 MB heap size)
PASS src/app/app.component.60.spec.ts (247 MB heap size)
PASS src/app/app.component.67.spec.ts (241 MB heap size)
PASS src/app/app.component.44.spec.ts (147 MB heap size)
PASS src/app/app.component.83.spec.ts (152 MB heap size)
PASS src/app/app.component.28.spec.ts (185 MB heap size)
PASS src/app/app.component.17.spec.ts (181 MB heap size)
PASS src/app/app.component.93.spec.ts (213 MB heap size)
PASS src/app/app.component.54.spec.ts (206 MB heap size)
PASS src/app/app.component.51.spec.ts (239 MB heap size)
PASS src/app/app.component.85.spec.ts (248 MB heap size)
PASS src/app/app.component.82.spec.ts (205 MB heap size)
PASS src/app/app.component.45.spec.ts (210 MB heap size)
PASS src/app/app.component.36.spec.ts (241 MB heap size)
PASS src/app/app.component.25.spec.ts (249 MB heap size)
PASS src/app/app.component.19.spec.ts (251 MB heap size)
PASS src/app/app.component.91.spec.ts (151 MB heap size)
PASS src/app/app.component.84.spec.ts (164 MB heap size)
PASS src/app/app.component.8.spec.ts (198 MB heap size)
PASS src/app/app.component.30.spec.ts (189 MB heap size)
PASS src/app/app.component.47.spec.ts (201 MB heap size)
PASS src/app/app.component.2.spec.ts (232 MB heap size)
PASS src/app/app.component.89.spec.ts (238 MB heap size)
PASS src/app/app.component.86.spec.ts (244 MB heap size)
PASS src/app/app.component.6.spec.ts (170 MB heap size)
PASS src/app/app.component.33.spec.ts (168 MB heap size)
PASS src/app/app.component.69.spec.ts (204 MB heap size)
PASS src/app/app.component.20.spec.ts (193 MB heap size)
PASS src/app/app.component.70.spec.ts (199 MB heap size)
PASS src/app/app.component.72.spec.ts (230 MB heap size)
PASS src/app/app.component.7.spec.ts (237 MB heap size)
PASS src/app/app.component.32.spec.ts (240 MB heap size)
PASS src/app/app.component.29.spec.ts (167 MB heap size)
PASS src/app/app.component.38.spec.ts (161 MB heap size)
PASS src/app/app.component.22.spec.ts (195 MB heap size)
PASS src/app/app.component.99.spec.ts (192 MB heap size)
PASS src/app/app.component.98.spec.ts (215 MB heap size)
PASS src/app/app.component.35.spec.ts (214 MB heap size)
Test Suites: 101 passed, 101 total
Tests: 101 passed, 101 total
Snapshots: 0 total
Time: 41.712 s
Ran all test suites.
@platov Thanks for looking into this.
Yes if you lower the max memory memory garbage collection will be triggered before it grows beyond the limit. However I think it is still leaking, checkout this article:
https://chanind.github.io/javascript/2019/10/12/jest-tests-memory-leak.html
The memory should be mostly constant especially for simple test like the ones in the sample repo.
setup-jest.js
I don't think is in my sample repo. It is in this jest-preset-angular
repo.
*I forked this sample repo FYI. If you can find the file for me that would be great.
Looks like you just changed it 18 minutes ago
https://github.com/thymikee/jest-preset-angular/pull/2163
I will retest and see if this has improved the situation.
@vespertilian, i agree that Jest has memory leak itself that is not related to jest-preset-angular. Details can be found in this ticket.
To solve Jest memory leak issue in my project helped usage of the --no-compilation-cache
flag for nodejs.
Try to run your test like in the following example:
node --no-compilation-cache node_modules/jest/bin/jest
setup-jest.js I don't think is in my sample repo. It is in this jest-preset-angular repo.
Sorry i was not clear. The bootstrap logic of Angular testing module is placed here. If You remove import 'jest-preset-angular/setup-jest';
line then sample tests will be executed faster.
@platov Thanks for this.
Yes moving out setup jest does speed it up, however you need setup jest if you want to test anything meaningful, like a component. I want to dig into this a bit more, I won't have time until later this week.
Thanks again. Stay tuned.
@vespertilian Did you find something that's worth to share? I'm in this rabbithole for a while now and was happy to find at least this issue.
What I've found out so far was updating to node 20.10.x brought some improvements but more as a positive side effect due to optimizations on "fs"
But all in all the tests feel much too slow for what's really happening and it's very time consuming in the hooks/pipeline. And it's impossible to execute on older laptops
@BernhardBehrendt
I have been looking at it again today. No luck yet. I will probably do a bit more on Monday or Tuesday.
Weirdly this
node --no-compilation-cache --expose-gc node_modules/jest/bin/jest --logHeapUsage
Seems to fix the issue in my test repo but you need --expose-gc ass well as --no-compilation-cache
Are you using NX or just the Angular CLI?
Hey @vespertilian I'm using Angular using NX and all in the latest version.
Edit: I've tried your provided command and it's more less the same as before. My Computer (MBPM2) gets so overloaded that even music starts to fragmentize.
@platov Thanks for this.
Yes moving out setup jest does speed it up, however you need setup jest if you want to test anything meaningful, like a component. I want to dig into this a bit more, I won't have time until later this week.
Thanks again. Stay tuned.
Weirdly I tried this suggestion and improved my memory times. Do you know the why ?!?!
@BernhardBehrendt
I have been looking at it again today. No luck yet. I will probably do a bit more on Monday or Tuesday.
Weirdly this
node --no-compilation-cache --expose-gc node_modules/jest/bin/jest --logHeapUsage
Seems to fix the issue in my test repo but you need --expose-gc ass well as --no-compilation-cache
Are you using NX or just the Angular CLI?
Weirdly I tried this suggestion and improved my memory times. Do you know the why @BernhardBehrendt ?!?!
Just made the upgrade to 14.0.0 and it's still no noticeable improvement.
@ricardomiguelmoura retried also
node --no-compilation-cache --expose-gc node_modules/jest/bin/jest --logHeapUsage
but it's totally "killing" my machine.
Have actually no more idea and hope that angular will move jest esbuild support soon out of experimental so that it bet better integation into nx.
@BernhardBehrendt but your problem is memory or cpu ? Because @vespertilian solution could ensure me that in regards to memory after some time i see that memory between tests isnt increasing so much as without the flags mentioned. It seems that it does garbage collection test after test ... it really improves in regards to memory while tests are running .. In my case helped me a lot
@ricardomiguelmoura it's still memory in my case. CPU is fine
I am experiencing this problem as well in my nx workspace with a mix of Angular and NestJS apps/libraries. During development it is hardly noticed that the tests leak memory. I am not using the most recent versions of any package used, but I have read multiple times that this issue prevailed through multiple versions of Angular and nx. My investigation is described in the following notes:
The CI pipeline executes all tests in the workspace by running nx run-many --all --target=test
. Eventually, the runners were hitting their memory limits and I began investigating their memory usage.
First step is to demonstrate the growing memory usage. We can execute jest directly, make it execute tests one at a time and log the heap usage:
./node_modules/.bin/jest --runInBand --logHeapUsage
For apps with many tests, the memory usage hits the limits of the pipeline runners but not my local machine, so i can verify for sure that the memory usage is way higher than it should be (several gigabytes at some point).
To narrow it down, I have an example app with just a couple of tests. It consistently shows increasing memory usage with some ups and downs of course.
./node_modules/.bin/jest apps/error-logging-app-frontend --runInBand --logHeapUsage
PASS error-logging-app-frontend apps/error-logging-app-frontend/src/app/modules/error-logs/pages/error-log-details-page/error-log-details-page.component.spec.ts (381 MB heap size)
PASS error-logging-app-frontend apps/error-logging-app-frontend/src/app/modules/error-logs/pages/error-logs-page/error-logs-page.component.spec.ts (377 MB heap size)
PASS error-logging-app-frontend apps/error-logging-app-frontend/src/app/modules/user-management/pages/user-management-page/user-management-page.component.spec.ts (448 MB heap size)
PASS error-logging-app-frontend apps/error-logging-app-frontend/src/app/app.component.spec.ts (499 MB heap size)
PASS error-logging-app-frontend apps/error-logging-app-frontend/src/app/modules/login/pages/login-page/login-page.component.spec.ts (565 MB heap size)
PASS error-logging-app-frontend apps/error-logging-app-frontend/src/app/modules/user-management/dialogs/create-user-dialog/create-user-dialog.component.spec.ts (626 MB heap size)
PASS error-logging-app-frontend apps/error-logging-app-frontend/src/app/modules/e2e/e2e-hidden-page/e2e-hidden-page.component.spec.ts (468 MB heap size)
Test Suites: 7 passed, 7 total
Tests: 7 passed, 7 total
Snapshots: 0 total
Time: 4.502 s, estimated 5 s
Ran all test suites matching /apps\/error-logging-app-frontend/i.
I read about similar problems online and found a couple of approaches that I all tried, most importantly running the garbage collection manually with --expose-gc
and adding afterAll(() => global.gc && global.gc())
to the global jest setup. These were good for confirming the issue, but did not help solving it, so I continued to narrow it down.
Next I ran the test command for a single test file:
./node_modules/.bin/jest apps/error-logging-app-frontend/src/app/app.component.spec.ts --runInBand --logHeapUsage
PASS error-logging-app-frontend apps/error-logging-app-frontend/src/app/app.component.spec.ts (357 MB heap size)
AppComponent
✓ should create the app (78 ms)
Test Suites: 1 passed, 1 total
Tests: 1 passed, 1 total
Snapshots: 0 total
Time: 1.012 s
Ran all test suites matching /apps\/error-logging-app-frontend\/src\/app\/app.component.spec.ts/i.
The test runs successfully. However, jest has the experimental feature --detectLeaks
:
./node_modules/.bin/jest apps/error-logging-app-frontend/src/app/app.component.spec.ts --runInBand --logHeapUsage --detectLeaks
FAIL error-logging-app-frontend apps/error-logging-app-frontend/src/app/app.component.spec.ts
● Test suite failed to run
EXPERIMENTAL FEATURE!
Your test suite is leaking memory. Please ensure all references are cleaned.
There is a number of things that can leak memory:
- Async operations that have not finished (e.g. fs.readFile).
- Timers not properly mocked (e.g. setInterval, setTimeout).
- Keeping references to the global scope.
at onResult (../../node_modules/@jest/core/build/TestScheduler.js:150:18)
at ../../node_modules/@jest/core/build/TestScheduler.js:254:19
at ../../node_modules/emittery/index.js:363:13
at Array.map (<anonymous>)
at Emittery.emit (../../node_modules/emittery/index.js:361:23)
Test Suites: 1 failed, 1 total
Tests: 0 total
Snapshots: 0 total
Time: 1.193 s
Ran all test suites matching /apps\/error-logging-app-frontend\/src\/app\/app.component.spec.ts/i.
This is an experimental feature of course, but at this point it is all I have available. For the next step, we will look at the test and try to locate the problem by eliminating code:
describe('AppComponent', () => {
beforeEach(async () => {
// await TestBed.configureTestingModule({
// imports: [RouterTestingModule],
// declarations: [AppComponent],
// }).compileComponents();
});
it('should create the app', () => {
// const fixture = TestBed.createComponent(AppComponent);
// const app = fixture.componentInstance;
// expect(app).toBeTruthy();
expect(true).toBeTruthy();
});
});
All Angular-related code is removed and the memory issue still exists. Next, I removed the jest-preset-angular
import from the test-setup.ts
file in the project directory:
// import 'jest-preset-angular/setup-jest';
Now the test succeeds without memory errors:
./node_modules/.bin/jest apps/error-logging-app-frontend/src/app/app.component.spec.ts --runInBand --logHeapUsage --detectLeaks
PASS error-logging-app-frontend apps/error-logging-app-frontend/src/app/app.component.spec.ts (291 MB heap size)
AppComponent
✓ should create the app (1 ms)
Test Suites: 1 passed, 1 total
Tests: 1 passed, 1 total
Snapshots: 0 total
Time: 0.408 s
Ran all test suites matching /apps\/error-logging-app-frontend\/src\/app\/app.component.spec.ts/i.
My result so far is: it seems that jest-preset-angular
inside my quite standard jest project causes the memory leaks.
For more information, I also tried debugging the memory usage of node directly with node --inspect-brk --expose-gc ./node_modules/.bin/jest /apps/error-logging-app-frontend/ --runInBand --logHeapUsage
and using the Chrome debugger tools. It revealed large memory allocations for angular source files, compiled code and so on. I was looking for patterns of repeated allocations of the same data, but in the tens of thousands of allocations to scroll through, I haven't found anything revealing yet.
I will gladly go into more detail on each of my steps, any help to reduce the memory usage is greatly appreciated.
@jpv-os I have narrowed down exactly to the same point before seeing your post actually. jest-preset-angular in the setup-jest will show the memory leak while using --detectLeaks. Did you find any solution?
Thanks!
I believe this is happening for me as well. I'm using the latest Nx release (19) and node (22) and every test that runs seems to get linearly slower. Or if I watch heap usage, it linearly grows. Are we going to see any progress on this anytime soon?
I’m working with Angular team on this https://github.com/thymikee/jest-preset-angular/pull/2615 which I think will be the recommended way to go.
@ahnpnl could you please elaborate why you think that it will fix the memory issue?
The PR would recommend projects to switch to use "transpilation" mode, which is related to the term transpiler
with typical examples like esbuild
and swc
. transpilation
mode would consume less memory and provide better feedback loops.
bump, we have a project that takes 10g+ of ram because of this issue.
@statop you can try to use isolatedModules
like this https://github.com/thymikee/jest-preset-angular/blob/0f98727e76b2491590225ad3609e8411724e5265/examples/example-app-v18/jest-isolated.config.js#L13 to see if it helps
I really don’t want to disable type checking. Will try it though.
We will try to add type checking to that mode via a new flag and enable by default in the next major version
isolatedModules didn't seem to really fix it. It made the run faster, but still used lots of memory.
@statop indeed, this is the downside of transpiling codes with typescript API.
One possible solution to solve memory issue better is switching to use swc
but this requires rewriting Angular AST transformers into Rust and maintain them.
@ahnpnl What would be required for rewriting the angular AST? Would something like https://github.com/dlvandenberg/tree-sitter-angular be helpful in that scenario?
@nb-midwestern there are 2 main AST transformers need to be rewritten in Rust to work with swc
I think the repo you gave only focuses on HTML part. The transformers we need focus on component source code part, mainly processing ts
codes.
It doesn't seem like that is the problem. It still just seems like a memory leak. Memory shouldn't "only go up" with each test. Perhaps create a new typescript compiler for each test.
I remember there is one issue in Jest repo about memory leak. This repo has limited scope to transpilation process only.
Each test runs with a standalone instance ts Program
. Under isolatedModules: true
, typescript api also creates a Program
. From my exp with ts Program
, it also eats quite some memory.
Idk how garbage collection works with Jest either. Jest supposes to free memory of each worker instance once it finishes the job.
@ahnpnl You've mentioned 2 AST transformers that need to be rewritten in rust, after that is done what would be the next step? It sounds like an interesting challenge, but I don't really know where to start, what would be the end goal? Are there any resources that you know of that I could reference?
@nb-midwestern Since swc
supports AST transformers in Rust, we can use swc
to transpile Angular codes with the 2 rewritten transformers. It will bring benefits of speed and memory efficiency.
Currently, we rely on typescript API to transpile codes, which is a lot slower than swc
. There are also other benefits of using swc
too. In the end, we should only need ts api for type checking (can be opted out).
Angular team is looking into approach to build everything and run tests based on build output. It is still in progress and I think we shouldn’t reinvent the wheel into that direction giving limited maintaining resources. The builder is at https://github.com/angular/angular-cli/blob/main/packages/angular_devkit/build_angular/src/builders/jest/index.ts
oh neat, I suppose we'll wait. I also found this library, I haven't tried it, but it looks promising for swc https://github.com/jscutlery/devkit/tree/main/packages/swc-angular-plugin
Oh yesss, that’s exactly what I am looking for. Feel free to investigate how to integrate it. I will try to take a look as well. Thanks!
I implemented it on my tests and saw only a slight preformance increase but it still eats through all of the memory
I see the instruction from swc-angular-plugin
still points to use jest-preset-angular
to process html
file. This is not yet ideal situation since jest-preset-angular
invokes ts-jest
to process and cache some stuffs. I think those can be improvement points.
I'm thinking about a rewritten the existing https://github.com/thymikee/jest-preset-angular/blob/main/src/ng-jest-transformer.ts which I think should improve further.
Version
13.1.0
Steps to reproduce
Clone repo
https://github.com/vespertilian/jest-preset-angular-mem-usage
npm install
npm run test-jest
Expected behavior
I would expect for simple tests memory to not increase at an alarming rate.
Actual behavior
Testing Angular seem to leak memory which makes the tests run really slow on older computers when you have multiple libraries
Some more info here NX
https://github.com/nrwl/nx/issues/18926
Thanks for all your had work on this!
Additional context
No response
Environment