nrwl / nx

Smart Monorepos · Fast CI
https://nx.dev
MIT License
23.62k stars 2.36k forks source link

nx test: Jest unit tests are failing to run in any test that has been imported from other libraries. #10100

Closed marimendez88 closed 2 years ago

marimendez88 commented 2 years ago

Current Behavior

An NX - Ionic and Vue workspace fails to run tests, many solutions tested and any seems to fix the issue:

This is the error message:

  ● Test suite failed to run

    Jest encountered an unexpected token

    Jest failed to parse a file. This happens e.g. when your code or its dependencies use non-standard JavaScript syntax, or when Jest is not configured to support such syntax.

    Out of the box Jest supports Babel, which will be used to transform your files into valid JS based on your Babel configuration.

    By default "node_modules" folder is ignored by transformers.

    Here's what you can do:
     • If you are trying to use ECMAScript Modules, see https://jestjs.io/docs/ecmascript-modules for how to enable it.
     • If you are trying to use TypeScript, see https://jestjs.io/docs/getting-started#using-typescript
     • To have some of your "node_modules" files transformed, you can specify a custom "transformIgnorePatterns" in your config.
     • If you need a custom transformation specify a "transform" option in your config.
     • If you simply want to mock your non-JS modules (e.g. binary assets) you can stub them out with the "moduleNameMapper" config option.

    You'll find more details and examples of these config options in the docs:
    https://jestjs.io/docs/configuration
    For information about custom transformations, see:
    https://jestjs.io/docs/code-transformation

Solutions tested:

Related Issues:

https://github.com/nrwl/nx/issues/8323 https://github.com/nrwl/nx/issues/7844 https://github.com/nrwl/nx/issues/8354 https://github.com/nrwl/nx/issues/9825 https://github.com/ionic-team/ionic-framework/issues/25223

Investigation made:

Expected Behavior

Steps to Reproduce

Here you can find a minimal repository to reproduce the issue:

https://github.com/marimendez88/of-jest-issue

  1. Create empty workspace: Run npx create-nx-workspace@^13.0.0
  2. select → apps option
  3. Install testing dependencies: npm install @nrwl/cypress@^13.0.0 @nrwl/jest@^13.0.0 @nrwl/linter@^13.0.0 --save-dev
  4. Install nx vue plugin : npm install @nx-plus/vue --save-dev
  5. Creating sample app: nx g @nx-plus/vue:app sample-app
  6. Install ionic npm i @ionic/vue npm i @ionic/vue-router
  7. Update main.ts under sample-app to use ionic
  8. Creating sample lib nx g @nx-plus/vue:lib sample-lib
  9. Create a header test component under sample-lib
  10. Import the header component into the sample app to use it
  11. Successfully served and components are shown as expected
  12. Run a test, error happens.

Failure Logs

This is the error message:

  ● Test suite failed to run

    Jest encountered an unexpected token

    Jest failed to parse a file. This happens e.g. when your code or its dependencies use non-standard JavaScript syntax, or when Jest is not configured to support such syntax.

    Out of the box Jest supports Babel, which will be used to transform your files into valid JS based on your Babel configuration.

    By default "node_modules" folder is ignored by transformers.

    Here's what you can do:
     • If you are trying to use ECMAScript Modules, see https://jestjs.io/docs/ecmascript-modules for how to enable it.
     • If you are trying to use TypeScript, see https://jestjs.io/docs/getting-started#using-typescript
     • To have some of your "node_modules" files transformed, you can specify a custom "transformIgnorePatterns" in your config.
     • If you need a custom transformation specify a "transform" option in your config.
     • If you simply want to mock your non-JS modules (e.g. binary assets) you can stub them out with the "moduleNameMapper" config option.

    You'll find more details and examples of these config options in the docs:
    https://jestjs.io/docs/configuration
    For information about custom transformations, see:
    https://jestjs.io/docs/code-transformation

    Details:

    /of-jest-issue/apps/sample-app/src/components/HelloWorld.vue:1
    ({"Object.<anonymous>":function(module,exports,require,__dirname,__filename,jest){import { defineComponent } from 'vue';
                                                                                      ^^^^^^

    SyntaxError: Cannot use import statement outside a module

      1 | import { shallowMount } from '@vue/test-utils';
    > 2 | import HelloWorld from '../../src/components/HelloWorld.vue';
        | ^
      3 |
      4 | describe('HelloWorld.vue', () => {
      5 |   it('renders props.msg when passed', () => {

      at Runtime.createScriptFromCode (../../node_modules/jest-runtime/build/index.js:1728:14)
      at Object.<anonymous> (tests/unit/example.spec.ts:2:1)

Test Suites: 1 failed, 1 total
Tests:       0 total
Snapshots:   0 total
Time:        5.015 s
Ran all test suites.

Environment

 >  NX   Report complete - copy this into the issue template

   Node : 14.17.1
   OS   : darwin x64
   npm  : 6.14.13

   nx : 13.10.3
   @nrwl/angular : Not Found
   @nrwl/cypress : 13.10.3
   @nrwl/detox : Not Found
   @nrwl/devkit : 13.10.3
   @nrwl/eslint-plugin-nx : 13.10.3
   @nrwl/express : Not Found
   @nrwl/jest : 13.10.3
   @nrwl/js : Not Found
   @nrwl/linter : 13.10.3
   @nrwl/nest : Not Found
   @nrwl/next : Not Found
   @nrwl/node : Not Found
   @nrwl/nx-cloud : Not Found
   @nrwl/nx-plugin : Not Found
   @nrwl/react : Not Found
   @nrwl/react-native : Not Found
   @nrwl/schematics : Not Found
   @nrwl/storybook : Not Found
   @nrwl/web : Not Found
   @nrwl/workspace : 13.10.3
   typescript : 4.6.4
   rxjs : 6.6.7
   ---------------------------------------
   Community plugins:
     @nx-plus/vue: 13.0.1
Marcelh1983 commented 2 years ago

Think I have the same error. I created a repository to reproduce starting from scratch with react. nx test gives the same SyntaxError: Cannot use import statement outside a module error. See https://github.com/Marcelh1983/nx-jest-error

barbados-clemens commented 2 years ago

I'm not sure what change you're looking for the nx team to make to fix this as this looks to be an issue with the nx-plus/vue plugin being use.

creating libs with @nrwl/* and testing them (and importing from each other) is working as expected in my testing. The error you're seeing is probably related to the nx-plug/vue plugin needing to be tweaked/updated as needed.

I noticed that the vue lib makes a babel.config.js file vs .babelrc file + root babel.config.json file that nx uses. this might be something you can look into to see if there is something there.

marimendez88 commented 2 years ago

hey @barbados-clemens , thank you for the response, I have been looking at this error on Angular and React projects as well, the Comment from @Marcelh1983 has a react repo that reproduces the issue, I think I can find another time by end of this week to provide another repo in an angular project as well.

PD: I have been looking into this on existing repos, I have tried to implement that the docs point but the example given is not clear, if I try to import this: import { registerTsProject } from 'nx/src/utils/register'; immediately see an issue that the nx/src/... doesn't exists.

Also, I followed the thread here as per your official issue closed 22 days ago, but, the repo is not accessible and the video doesn't seem to help me either.

barbados-clemens commented 2 years ago

I am rebuilding a workspace to double-check all of this.

but in the meantime can you try using require syntax for imports? we are running the ts project register in our e2e tests so that should work, but I just looked at them and they are using const {} = require() instead of import {} from ''

but that should only be needed for the global setup/teardown as ts paths are available at that point to jest. but within a test they should mapped already.

barbados-clemens commented 2 years ago

update using libs within another lib works just fine in testing for both angular and react.

only thing I ran into is the global setup/teardown with react seems to error out. so checking on that but the same setup/teardown works just fine in angular 🤔

will share repo when I get a chance

Marcelh1983 commented 2 years ago

Hi @barbados-clemens thanks for checking and helping! Can you reproduce the error I got with a new nx setup and a jest test using an external library. In my case the error occurs when I'm using an external component from node_modules that contains an import I'm not sure if it's the same error/cause as @marimendez88. repo

barbados-clemens commented 2 years ago

@Marcelh1983 yes it works I use MUI for react and material for angular Here is the repo showing it working. only thing not working is the global setup/teardown in the react lib one.

to test run npx nx test r-lib-one or npx nx test ng-lib-one the *-lib-one consumes the *-lib-two version of that lib and the *-lib-two lib also consumes a UI component framework from node_modules.

https://github.com/barbados-clemens/using-libs-with-nx-demo

Marcelh1983 commented 2 years ago

I added my test in your repos and it fails with the same exception. Could have something todo with the package it use.

`C:\TMP\testjest\node_modules\rx-basic-store\index.js:1
({"Object.<anonymous>":function(module,exports,require,__dirname,__filename,jest){import { BehaviorSubject } from 'rxjs';
                                                                                  ^^^^^^

SyntaxError: Cannot use import statement outside a module

> 1 | import { ActionType, createStore, StateContextType } from 'rx-basic-store';
    | ^
  2 |
  3 | export interface StateModel {
  4 |   loaded: boolean;

  at Runtime.createScriptFromCode (../../node_modules/jest-runtime/build/index.js:1728:14)`
barbados-clemens commented 2 years ago

I was able to get the data lib to pass via adding

   transformIgnorePatterns: ["/node_modules/(?!rx-basic-store)"],

to the jest.config.ts and then allowJs: true in the tsconfig.

the testjest app still seems to be having issues even with the transformIgnorePatterns as the app.spec.tsx is the issue there. which should be picked up via the ts-jest via the tsconfig.spec.json which is odd.

Marcelh1983 commented 2 years ago

@barbados-clemens thanks, That works!

matt-forster commented 2 years ago

I can reproduce this with a new react workspace; following https://nx.dev/react-tutorial/01-create-application. Not even modifying any of the files; I added the transform ignore and allowJs flag to ./jest.config.ts and ./tsconfig.base.json, but I still get the same error.

  1. npx create-nx-workspace@latest
  2. Select react, less
  3. run nx test

https://nx.app/runs/rXGssNRGlFG

➜ nx test   

> nx run site:test

 FAIL   site  apps/site/src/app/app.spec.tsx
  ● Test suite failed to run

    Jest encountered an unexpected token

    Jest failed to parse a file. This happens e.g. when your code or its dependencies use non-standard JavaScript syntax, or when Jest is not configured to support such syntax.

    Out of the box Jest supports Babel, which will be used to transform your files into valid JS based on your Babel configuration.

    By default "node_modules" folder is ignored by transformers.

    Here's what you can do:
     • If you are trying to use ECMAScript Modules, see https://jestjs.io/docs/ecmascript-modules for how to enable it.
     • If you are trying to use TypeScript, see https://jestjs.io/docs/getting-started#using-typescript
     • To have some of your "node_modules" files transformed, you can specify a custom "transformIgnorePatterns" in your config.
     • If you need a custom transformation specify a "transform" option in your config.
     • If you simply want to mock your non-JS modules (e.g. binary assets) you can stub them out with the "moduleNameMapper" config option.

    You'll find more details and examples of these config options in the docs:
    https://jestjs.io/docs/configuration
    For information about custom transformations, see:
    https://jestjs.io/docs/code-transformation

    Details:

    /home/matt/personal/website/apps/site/src/app/app.spec.tsx:4
    import { render } from '@testing-library/react';
    ^^^^^^

    SyntaxError: Cannot use import statement outside a module

      at Runtime.createScriptFromCode (../../node_modules/jest-runtime/build/index.js:1728:14)
      at TestScheduler.scheduleTests (../../node_modules/@jest/core/build/TestScheduler.js:333:13)

Test Suites: 1 failed, 1 total
Tests:       0 total
Snapshots:   0 total
Time:        0.564 s
Ran all test suites.
➜ nx report                      

 >  NX   Report complete - copy this into the issue template

   Node : 16.14.2
   OS   : linux x64
   npm  : 8.5.0

   nx : 14.1.4
   @nrwl/angular : Not Found
   @nrwl/cypress : 14.1.4
   @nrwl/detox : Not Found
   @nrwl/devkit : 14.1.4
   @nrwl/eslint-plugin-nx : 14.1.4
   @nrwl/express : Not Found
   @nrwl/jest : 14.1.4
   @nrwl/js : 14.1.4
   @nrwl/linter : 14.1.4
   @nrwl/nest : Not Found
   @nrwl/next : Not Found
   @nrwl/node : Not Found
   @nrwl/nx-cloud : 14.0.3
   @nrwl/nx-plugin : Not Found
   @nrwl/react : 14.1.4
   @nrwl/react-native : Not Found
   @nrwl/schematics : Not Found
   @nrwl/storybook : 14.1.4
   @nrwl/web : 14.1.4
   @nrwl/workspace : 14.1.4
   typescript : 4.6.4
   rxjs : 6.6.7
   ---------------------------------------
   Community plugins:
barbados-clemens commented 2 years ago

Just tried it and the default tests pass. didn't need to change anything. so I'm not sure what issues you're running into. Can you share the repo @matt-forster ? https://nx.app/runs/vbseUwO2T5w


 >  NX   Report complete - copy this into the issue template

   Node : 16.15.0
   OS   : darwin arm64
   npm  : 8.5.5

   nx : 14.1.4
   @nrwl/angular : Not Found
   @nrwl/cypress : 14.1.4
   @nrwl/detox : Not Found
   @nrwl/devkit : 14.1.4
   @nrwl/eslint-plugin-nx : 14.1.4
   @nrwl/express : Not Found
   @nrwl/jest : 14.1.4
   @nrwl/js : 14.1.4
   @nrwl/linter : 14.1.4
   @nrwl/nest : Not Found
   @nrwl/next : Not Found
   @nrwl/node : Not Found
   @nrwl/nx-cloud : 14.0.3
   @nrwl/nx-plugin : Not Found
   @nrwl/react : 14.1.4
   @nrwl/react-native : Not Found
   @nrwl/schematics : Not Found
   @nrwl/storybook : 14.1.4
   @nrwl/web : 14.1.4
   @nrwl/workspace : 14.1.4
   typescript : 4.6.4
   rxjs : 6.6.7
   ---------------------------------------
   Community plugins:
matt-forster commented 2 years ago

@barbados-clemens - Yep.

Vanilla (no changes): https://github.com/matt-forster/website-new/tree/6603d3901505c81a15e76fa9e9270735cd2e8b77 With transform attempt: https://github.com/matt-forster/website-new/commit/27c28cfdd2d682f97d6f5db682d615332263d662

Steps:

  1. Clone
  2. npm i
  3. nx test
barbados-clemens commented 2 years ago

it works for me @matt-forster https://nx.app/runs/P6QjhYlCgaY working tests for example repo

matt-forster commented 2 years ago

@barbados-clemens Huh - It works the first time, but then as soon as I change something, it starts failing. If you change a file (I changed package.json, and in a different test, a source file), does it start failing?

Package Change:

File Change:

barbados-clemens commented 2 years ago

this is pretty wild @matt-forster as doing the exact same thing works for me. the only thing I could assume here is the different OS. It looks like you're on linux while I'm on macOS.

https://share.cleanshot.com/HkDHjx

matt-forster commented 2 years ago

WSL2 VM from Win 11;

➜ bat  /etc/*-release
───────┬──────────────────────────────────────────────────────────────────────────────────────────
       │ File: /etc/lsb-release
       │ Size: 104 B
───────┼──────────────────────────────────────────────────────────────────────────────────────────
   1   │ DISTRIB_ID=Ubuntu
   2   │ DISTRIB_RELEASE=20.04
   3   │ DISTRIB_CODENAME=focal
   4   │ DISTRIB_DESCRIPTION="Ubuntu 20.04.4 LTS"
───────┴──────────────────────────────────────────────────────────────────────────────────────────
───────┬──────────────────────────────────────────────────────────────────────────────────────────
       │ File: /etc/os-release
       │ Size: 382 B
───────┼──────────────────────────────────────────────────────────────────────────────────────────
   1   │ NAME="Ubuntu"
   2   │ VERSION="20.04.4 LTS (Focal Fossa)"
   3   │ ID=ubuntu
   4   │ ID_LIKE=debian
   5   │ PRETTY_NAME="Ubuntu 20.04.4 LTS"
   6   │ VERSION_ID="20.04"
   7   │ HOME_URL="https://www.ubuntu.com/"
   8   │ SUPPORT_URL="https://help.ubuntu.com/"
   9   │ BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
  10   │ PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
  11   │ VERSION_CODENAME=focal
  12   │ UBUNTU_CODENAME=focal
───────┴──────────────────────────────────────────────────────────────────────────────────────────
barbados-clemens commented 2 years ago

I'm going to ping some other people on the nx team that have windows machines to see if they can take a look if it is the OSes causing the issue. I'd be surprised if it was but it's the only thing different that I can see. if it is. I imagine it might be a jest issue vs nx set up issue, but I'll have to dig into it if we can reproduce it via WSL.

Xjph commented 2 years ago

Was able to reproduce as described above, WSL2 on Win10. Test was initially passing, but after modifying apps/site/src/app/app.tsx in what should be inconsequential ways (adding a comment, or a second empty div after the title) the tests fail. Reverting the change restores the tests to passing. image image

matt-forster commented 2 years ago

I've talked to some co-workers using the same or similar env, and they can reproduce it. It is widely inconsistent though.

Other info; using Volta as node manager, nx installed globally using Volta. Other than that I don't think anything else would even remotely affect it.

It might have something to do with the lock, I committed some more changes (including the updated lock after first install) and now the tests fail immediately for me after a fresh clone.

barbados-clemens commented 2 years ago

the reason the tests pass initially is more than likely the nx cloud cache just an FYI. you can check as it'll say remote cache in the results @Xjph

marimendez88 commented 2 years ago

Hey @barbados-clemens , I have been working on a solution for this and I was able to fix most of the issue, But I still have an issue with NX libraries. I followed your recommendation to create libraries using the @nrwl/* ..

Created a PR (https://github.com/marimendez88/of-jest-issue/pull/2 / branch test/may-20) with the changes to the repo I posted earlier:

  1. There were some issues with versions so I migrated everything to the latest version and worked well.

  2. Create a second library ( npx nx generate @nrwl/workspace:library second-library-nx --importPath=@of-jest-issue/second-library-nx --pascalCaseFiles --testEnvironment=node )

  3. This created the normal web lib and the ts tests worked.

  4. Adapted the library to vue: Create a small component and it's test.

  5. Modify jest.config.js config to add the following:

    transform: {
    '^.+\\.[tj]sx?$': 'ts-jest',
    '^.+.vue$': 'vue3-jest',
    }, 
    transformIgnorePatterns: ['/node_modules/(?!@ionic/core|@stencil/core|ionicons)'],
  6. Run tests : Pass Screen Shot 2022-05-23 at 10 47 46

  7. Create the third library to tests nx paths within test: npx nx generate @nrwl/workspace:library third-library-nx --importPath=@of-jest-issue/third-library-nx --pascalCaseFiles NOTE: did not setup the test env to node as the previous.

  8. Did the last steps to add vue component and test.

  9. Importe the thirdLib component into the second lib

  10. works well in the render (able to see the components while serving).

  11. Run tests on vue component on third libraries: PASS Screen Shot 2022-05-23 at 11 43 00

  12. Run tests on second library

  13. Error happened:

    > nx run second-library-nx:test
    
    PASS   second-library-nx  libs/second-library-nx/src/lib/SecondLibraryNx.spec.ts
    FAIL   second-library-nx  libs/second-library-nx/src/lib/SecondTest.spec.ts
    ● Test suite failed to run
    
    Cannot find module '@of-jest-issue/third-library-nx' from 'src/lib/SecondTest.vue'
    
    Require stack:
      src/lib/SecondTest.vue
      src/lib/SecondTest.spec.ts
    
      10 |
      11 | <script lang="ts">
    > 12 | import {ThirdComponent} from '@of-jest-issue/third-library-nx';
         | ^
      13 | import { defineComponent } from 'vue';
      14 | import {IonPage, IonHeader, IonTitle} from '@ionic/vue';
      15 | export default defineComponent({
    
      at Resolver.resolveModule (../../node_modules/jest-resolve/build/resolver.js:324:11)
      at Object.<anonymous> (src/lib/SecondTest.vue:12:1)

So the issue is still that jest doesn't recognize NX Libraries Paths, the 2 tests are able to see the actual VUEcomponent, The error comes when using libraries

marimendez88 commented 2 years ago

Hey, Just wanted to let you know I found a workaround, is working pretty well so far, let's see how it goes in the next weeks when I start creating new tests ;) I just posted what I did here:

https://www.marimendez88.dev/posts/nx-vue-jest-support and medium: https://medium.com/@marimendez88/nx-vue-jest-support-666156038122

BePasquet commented 2 years ago

i think there is an issue with the version on @barbados-clemens repo the test works, after runing nx migrate latest stops working

Screen Shot 2565-06-19 at 18 28 49 Cannot find module '@using-libs-with-jest/r-lib-two' from 'src/lib/r-lib-one.tsx'

BePasquet commented 2 years ago

i think there is an issue with the version on @barbados-clemens repo the test works, after runing nx migrate latest stops working

Screen Shot 2565-06-19 at 18 28 49 Cannot find module '@using-libs-with-jest/r-lib-two' from 'src/lib/r-lib-one.tsx'

Maybe this is usefull i downgraded to this versions and error is not happening any more

"@nrwl/cli": "14.1.1",
"@nrwl/jest": "14.1.1",
"@nrwl/workspace": "14.1.1",
"@testing-library/react": "13.1.1",
"react-test-renderer": "18.1.0"
barbados-clemens commented 2 years ago

@BePasquet can you check your root jest.config.ts and make sure you have

import { getJestProjects } from '@nrwl/jest';

export default {
  projects: getJestProjects(),
};

and in your root jest.preset.js

const nxPreset = require('@nrwl/jest/preset').default; // note the .default here!

module.exports = { ...nxPreset };
BePasquet commented 2 years ago

Hi @barbados-clemens i updated the versions, added that code to the config and preset and is working thanks a lot for the help

"@nrwl/cli": "14.3.6",
"@nrwl/jest": "14.3.6",
"@nrwl/workspace": "14.3.6",
"@testing-library/react": "13.3.0",
"react-test-renderer": "18.2.0"
corysmc commented 2 years ago

Here's what I had to do to get jest tests working after updating to ionic v6 using react in an nx project:

// <root>/apps/<app-name>/jest.config.js

const esModules = [
  '@ionic/core',
  '@stencil/core',
  'ionicons',
  '@ionic/react',
  '@ionic/react-router',
  '@ionic/react-test-utils',
].join('|');

module.exports = {
  displayName: 'tn-app',
  preset: '../../jest.preset.js',
  transform: {
    '^(?!.*\\.(js|jsx|ts|tsx|css|json)$)': '@nrwl/react/plugins/jest',
    '^.+\\.[tj]sx?$': 'ts-jest',
  },
  transformIgnorePatterns: [
    `/node_modules/(?!${esModules})`,
    '/node_modules/@stencil/core/internal/app-data',
  ],
  moduleFileExtensions: ['ts', 'tsx', 'js', 'jsx'],
  coverageDirectory: '../../coverage/apps/tn-app',
};

Update tsconfig to target es6:

// <root>/apps/<app-name>/tsconfig.json
{
  "extends": "../../tsconfig.base.json",
  "compilerOptions": {
    "jsx": "react-jsx",
    "allowJs": true,
    "skipLibCheck": true,
    "target": "es6",
    "lib": ["dom", "dom.iterable", "esnext", "es2017.object"],
    "esModuleInterop": true,
    "allowSyntheticDefaultImports": true,
    "strict": true,
    "forceConsistentCasingInFileNames": true,
    "module": "esnext",
    "moduleResolution": "node",
    "resolveJsonModule": true,
    "isolatedModules": true,
    "noEmit": true
  },
  "include": ["src", "src/**/*.d.ts"],
  "references": [
    {
      "path": "./tsconfig.app.json"
    },
    {
      "path": "./tsconfig.spec.json"
    }
  ]
}

Did the same thing for libs in our nx monorepo

github-actions[bot] commented 1 year ago

This issue has been closed for more than 30 days. If this issue is still occuring, please open a new issue with more recent context.