[Bug]: Huge performance drop when "allowJs": true is enabled in tsconfig

LoganTann commented 6 months ago

Version

at least since 0.3.3, tried in 0.5.3 and looks even worse (no proofs though)

Steps to reproduce

Minimal reproduction repository : https://github.com/LoganTann/tsjest-performance-issue-reproduction

Clone the repo
npm i
npm run test

Expected behavior

Jest should take 8 seconds to load the dependency, not more than 10 minutes.

Actual behavior

Simple test is taking an extremely long time (> 10min) to load the dependency
Disabling the {allowJs} property in the tsconfig file will make the test run under 8 seconds. However, jest will print over 1400 warnings (executed grep " file to compile while" output_nojs.log | wc -l).
After a successful run, later tests are expected to be much faster thanks to the caching.

Debug log

Interrupted execution, took too much time.

Additional context

Running in WSL 2, inside the ~/reproduction/ folder
No performance issues in the back-end, both in dev and prod. The code compiles and runs fine.
SWC + vitest is not affected by this issue, but I cannot migrate the whole app in another tech stack.

Performance measures I did five months ago (under 0.3.3)

``` cross-env NODE_ENV=test jest -runInBand --maxWorkers=1 SalesforceConnections.controller.spec.ts --no-cache Imports : HttpService: 62226 ms (@nestjs/axios) BadRequestException: 1 ms ConfigModule: 4736 ms (@nestjs/config) Test: 22756 ms (@nestjs/testing) TypeOrmModule: 133894 ms (@nestjs/typeorm) Repository: 0 ms configuration: 9 ms RequestUserModel: 8 ms CryptoService: 33 ms entities: 153 ms FixtureService: 109 ms TenantEntity: 0 ms TenantModel: 5 ms TenantService: 34 ms UserEntity: 0 ms SalesforceConnectionsController: 183104 ms (imports the same) SalesforceInstanceDetailsResponseDto: 0 ms SalesforceConnectionEntity: 0 ms SalesforceInstanceEntity: 0 ms SalesforceConnectionService: 0 ms SalesforceInstanceService: 0 ms SalesforceOauthService: 0 ms ConnectivityStatus: 1 ms Test module compilation : Compile: 11315 ms ```

Environment

System:
    OS: Linux 5.15 Ubuntu 22.04.3 LTS 22.04.3 LTS (Jammy Jellyfish)
    CPU: (8) x64 11th Gen Intel(R) Core(TM) i7-1165G7 @ 2.80GHz
  Binaries:
    Node: 18.19.0 - ~/.nvm/versions/node/v18.19.0/bin/node
    npm: 10.2.3 - ~/.nvm/versions/node/v18.19.0/bin/npm
  npmPackages:
    jest: ^29.7.0 => 29.7.0

LoganTann commented 6 months ago

Related: #1115

LoganTann commented 6 months ago

Found a workaround, suggested by the warning message when you disable "allowJs" :

Remove "compilerOptions.allowJs": true in your tsconfig file
Edit the transform regex to select only typescript files with ts-jest :

'transform': {
    '^.+\\.ts$': 'ts-jest', // << selects only ts. It was previously '^.+\\.(j|t)s$'
},

This is still an issue. This needs to be documented, as I had this issue since at least january and I had to investigate since I cleared my jest cache.

ahnpnl commented 4 months ago

A bit background for this issue: ts-jest internally uses TypeScript compiler API to transform ts/js file into js file.

In general, setting allowJs to true is necessary if one wants the TypeScript compiler API to load and process JavaScript files in addition to TypeScript files when transforming files. This leads to the result that more files are loaded into memory which can blow up the machine.

Currently there are 2 workarounds:

Use https://kulshekhar.github.io/ts-jest/docs/getting-started/options/isolatedModules
Or your solution which tells ts-jest not to process js file

With the release of typescript 5.5, now we can actually use other transpilers like esbuild or swc to process js files + opt in type checking from new v5.5 API

LoganTann commented 4 months ago

Good to know, thanks for the answer @ahnpnl ! Isolated modules did not solve the issue for me, but removing js files from ts-jest fixed it.

I just created another issue here : https://github.com/thisismydesign/nestjs-starter/issues/468 It is very likely the previous developer in my project took the ts-config file in this template repo.

ahnpnl commented 4 months ago

What is the issue with isolatedModules that you had? It supposed to be faster with true value (this is not tsconfig option btw, the name is confusing and we will rename it)

LoganTann commented 4 months ago

The reproduction repo has the isolatedModules set to true and my measurements takes account of this.

https://github.com/LoganTann/tsjest-performance-issue-reproduction/blob/main/jest.config.js

LoganTann commented 4 months ago

But the slowness is only due to js files being included in the transformer regex. isolatedModules does not seems to have any issue.

ahnpnl commented 4 months ago

I see. Yes in this case Jest will give both js and ts to ts-jest to transform which explained the issue.

Theoretically if using fast transpiler like swc, it should be better. However, the recommendation is still only transform what is needed.

LoganTann commented 4 months ago

The recommendation is still only transform what is needed.

For sure ! In fact, to solve this issue, only a simple warning in the documentation would be required. We cannot do much to solve this, as in order to get this behavior, you need a configuration that is not recommended.

ahnpnl commented 4 months ago

I agree, I will update documentation to have a note about this. Thanks for the suggestion!

LoganTann commented 1 month ago

Created a documentation PR to fix this : https://github.com/kulshekhar/ts-jest/pull/4567

sneko commented 1 week ago

Thank you all for the solutions. Just to give some insights about what it can improve:

I had a few test files taking more than 10 minutes to process (no cache)
From here, I wanted to "benchmark" by running tests only a specific test file (no cache): 4 minutes
Same test file, I switched my regexp '\\.[jt]s$' to '\\.ts$' as advised by the new doc (no cache): 10 seconds
Same test file, I added the isolatedModules: true that is not taken from my tsconfig.json as advised by the new doc (no cache): 5 seconds
Same test file, I enabled caching, so after 2+ retries it takes no more than: 3 seconds

@ahnpnl I think the documentation could both mention:

It's safe to disable ts-jest type checking if you put into your CI/CD a linter steps that does include .spec.ts files
To have the default enabled cache working In the CI/CD, some configurations must be done to persist cache across runs depending on the platform (GitHub Actions, GitLab Pipeline...)

kulshekhar / ts-jest