Unable to use services exported under a namespace w/ Jest 27 + Ng12

AgentEnder commented 3 years ago

🐛 Bug Report

Attempting to test components that inject a service imported from a namespace fails in Jest 27 / Angular 12.

To Reproduce

ng new my-app
Install jest / jest-preset-angular
Create a new folder, services.
Create a file in that folder (my-service.ts) containing the following:
```
import { Injectable } from '@angular/core';
```

@Injectable({ providedIn: 'root' }) export class MyService { constructor() { console.log('HELLO') } }

5. Create a new file (index.ts) inside the services folder containing the following:
```typescript
import * as Services from './my-service';

export { Services }

Add private myService: Services.MyService to the constructor of app-component
Import Services in app-component.
Try to run tests with npx jest

Expected behavior

Tests run successfully

Link to repo (highly encouraged)

https://github.com/AgentEnder/ng-jest-issue-6097

Error log:

 Can't resolve all parameters for AppComponent: (?).

      at syntaxError (../packages/compiler/src/util.ts:108:17)
      at CompileMetadataResolver.Object.<anonymous>.CompileMetadataResolver._getDependenciesMetadata (../packages/compiler/src/metadata_resolver.ts:1010:27)
      at CompileMetadataResolver.Object.<anonymous>.CompileMetadataResolver._getTypeMetadata (../packages/compiler/src/metadata_resolver.ts:889:20)
      at CompileMetadataResolver.Object.<anonymous>.CompileMetadataResolver.getNonNormalizedDirectiveMetadata (../packages/compiler/src/metadata_resolver.ts:387:18)
      at CompileMetadataResolver.Object.<anonymous>.CompileMetadataResolver.loadDirectiveMetadata (../packages/compiler/src/metadata_resolver.ts:238:41)
      at ../packages/compiler/src/jit/compiler.ts:137:36
          at Array.forEach (<anonymous>)
      at ../packages/compiler/src/jit/compiler.ts:135:65
          at Array.forEach (<anonymous>)
      at JitCompiler.Object.<anonymous>.JitCompiler._loadModules (../packages/compiler/src/jit/compiler.ts:132:71)
      at JitCompiler.Object.<anonymous>.JitCompiler._compileModuleAndAllComponents (../packages/compiler/src/jit/compiler.ts:117:32)
      at JitCompiler.Object.<anonymous>.JitCompiler.compileModuleAndAllComponentsAsync (../packages/compiler/src/jit/compiler.ts:69:33)
      at CompilerImpl.Object.<anonymous>.CompilerImpl.compileModuleAndAllComponentsAsync (../packages/platform-browser-dynamic/src/compiler_factory.ts:69:27)
      at TestingCompilerImpl.Object.<anonymous>.TestingCompilerImpl.compileModuleAndAllComponentsAsync (../packages/platform-browser-dynamic/testing/src/compiler_factory.ts:59:27)
      at TestBedViewEngine.Object.<anonymous>.TestBedViewEngine.compileComponents (../packages/core/testing/src/test_bed.ts:366:27)
      at Function.Object.<anonymous>.TestBedViewEngine.compileComponents (../packages/core/testing/src/test_bed.ts:155:25)
      at src/app/app.component.spec.ts:10:8
      at ZoneDelegate.Object.<anonymous>.ZoneDelegate.invoke (node_modules/zone.js/bundles/zone-testing-bundle.umd.js:407:30)
      at ProxyZoneSpec.Object.<anonymous>.ProxyZoneSpec.onInvoke (node_modules/zone.js/bundles/zone-testing-bundle.umd.js:3765:43)
      at ZoneDelegate.Object.<anonymous>.ZoneDelegate.invoke (node_modules/zone.js/bundles/zone-testing-bundle.umd.js:406:56)
      at Zone.Object.<anonymous>.Zone.run (node_modules/zone.js/bundles/zone-testing-bundle.umd.js:167:47)
      at Object.wrappedFunc (node_modules/zone.js/bundles/zone-testing-bundle.umd.js:4250:34)

envinfo

System:
    OS: Ubuntu (tested under wsl2)

Npm packages:
    jest: 27.0.5
    jest-preset-angular: 9.0.4
    typescript: 4.2.3

wtho commented 3 years ago

Does the same setup work in Angular v11 with same jest/ts-jest/jest-preset-angular versions?

ahnpnl commented 3 years ago

I debugged and saw that the compiled output of AppComponent contains undefined as ctor parameter instead of referencing the DemoService, which causes the issue. The problem could be that TypeScript LanguageService couldn't resolve the import from export namespace.

ahnpnl commented 3 years ago

The bug should occur to Angular 11 too as we use the same transformer.

One unknown thing is why Karma + Jasmine works. The most suspicious point would be module resolution doesn’t work correctly which makes LanguageService not able to find the information of the file.

Workaround

For now pls avoid using export/import namespace but following what Angular library does, e.g.

export { something } from ‘a-path’

Maximaximum commented 3 years ago

I'm facing exactly the same issue! (And it took me a few days to trace it down!).

The issue only happens with Jest (not with Jasmine/Karma) and only if using namespace imports, ie import * as Exported from './exported';.

In my case the issue is that I'm using some code generated by a 3rd party code generator that contains namespace imports, and there's no way to tweak the code generator's behavior

Maximaximum commented 3 years ago

And btw, if you run npx ngcc, the error message (Can't resolve all parameters for AppComponent: (?).) gets replaced with

This constructor is not compatible with Angular Dependency Injection because its dependency at index 0 of the parameter list is invalid.
    This can happen if the dependency type is a primitive like a string or if an ancestor of this class is missing an Angular decorator.

    Please check that 1) the type for the parameter at index 0 is correct and 2) the correct Angular decorators are defined for this class and its ancestors.

And I'm using Angular 12, not sure if same happens with Angular 11

Maximaximum commented 3 years ago

Here's a simplified reproduction repo: https://github.com/Maximaximum/jest-angular-namespace-import-bug

Maximaximum commented 3 years ago

I've just checked and the issue is reproducible with Angular v11 as well.

@thymikee @ahnpnl @wtho Is there anything I could do to help resolve this issue?

The suggested workaround is not usable for me, so this bug is blocking me from adding unit tests to my app. If this can't be fixed soon, I'll probably have to switch back to Jasmine+Karma.

ahnpnl commented 3 years ago

the only workaround is don't use import namespace for now but you should import directly from the file as well as avoiding barrel file because that won't work unfortunately.

We don't reuse the way how Angular CLI compiles codes therefore some efforts need to check.

One thing I haven't tested is: import namespace into a dummy file and rexport whatever comes from that namespace to import into component. The error occurs because import namespace is used directly in a file which contains Angular decorators.

wtho commented 3 years ago

@ahnpnl do you know why this importing is a problem? Is it related to ts-jest, jest or node?

ahnpnl commented 3 years ago

It is ts-jest problem as well as architecture problem. The error is caused by downlevel ctor transformer that it can’t resolve the injected dependencies which it modifies wrongly the AST.

I think this might be fixed if the LanguageService has the dependencies information to provide to the downlevel ctor transformer. However, this is still a problem with isolatedModules: true though because that mode does simple transpilation from ts to js.

The ideal way is: we follow completely the way like Angular CLI does. We would need to have a single place where the compilation is done, not at Jest transformer level.

Maximaximum commented 3 years ago

@ahnpnl As I have mentioned above, in some cases the workaround is not an option at all

ahnpnl commented 3 years ago

There is still one more workaround is: use ngc to compile everything in your project and point Jest to run on the output folder, similar to the approach of using tsc to compile everything and run Jest on output folder. For watch mode, that might not work.

Unfortunately we don't have a quick fix now so those 2 workarounds are the ones I can think of.

pheinicke commented 3 years ago

Hi, I have the exact same issue as @Maximaximum. That is Angular 12, Jest and code generated by a third party tool, containing namespace imports. @ahnpnl How would you tell Jest to run on the compiled files?

Thanks for the help!

ahnpnl commented 3 years ago

You can configure where Jest should look for the files by using testMatch, testRegex, rootDir. I think mainly rootDir, see https://jestjs.io/docs/configuration#rootdir-string

You have to use ngc to compile, don’t use tsc

Maximaximum commented 3 years ago

Fwiw, in the title of this issue "exported" should be replaced with "imported", because it's not the export syntax, but the import syntax that causes the issue

Maximaximum commented 3 years ago

FWIW I'm now trying to use the 2nd workaround suggested by @ahnpnl. But what drastically complicates things even more is that I'm having an nx workspace with about 20 different nx projects, instead of just a single Angular project. There doesn't seem to be a way to run ngc for every nx project automatically. And even if there was, I'm not sure where does the ngc output go, and how to make sure that jest is run against the built files, not against the sources.

Looks like I'm stuck with adding any unit tests to my project right now. Neither applying a workaround for jest, nor reverting back to using karma seems to be an easy thing to do.

ahnpnl commented 3 years ago

ngc can be configured but you would need to find documentation online.

Maximaximum commented 3 years ago

I can't find documentation about configuring the output folder here https://angular.io/guide/angular-compiler-options. Does ng build do the same thing as ngc? Should we run jest against the bundled files located in the dist folder, produced by ng build?

ahnpnl commented 3 years ago

IIRC ng build does similar thing like ngc. ngc is a replacement of tsc which does some Angular things extra.

After producing build outputs, you would need to configure Jest to run on output folder yes.

ahnpnl commented 3 years ago

Hi all, I found an easy workaround for this issue. You can configure Jest moduleNameMapper to instruct Jest to load the correct module. With the example repo from @AgentEnder, the configuration will be

// jest.config.js
module.exports = {
   moduleNameMapper: {
       './services$': '<rootDir>/src/app/services/demo-service.ts'
   }
}

There are 2 more possible workarounds:

Write a custom Jest resolver to desugar namespace import to be specific imports
Use path-mapping AST transformer from ts-jest https://kulshekhar.github.io/ts-jest/docs/getting-started/options/astTransformers#public-transformers (you might run into error if the codes use ESM export syntax). For example:
```
// jest.config.js
module.exports = {
globals: {
'ts-jest': {
  astTransformers: {
    before: ['ts-jest/dist/transformers/path-mapping']
  }
}
},
}
```

Maximaximum commented 3 years ago

@ahnpnl I don't know about the exact use cases of the other folks here, but in my specific case I have dozens of auto-generated files containing namespaced imports. Adding an entry to jest.config.js for each of these imports is definitely not an option, because I'm having an nx workspace with about 20 different angular projects, and each of them having its own jest.config.js. Managing all these moduleNameMapper settings in all of the jest.config.js would be a nightmare!

Creating a custom resolver might be an option, but I'm totally new to Jest, so it might be quite an overwhelming task for me.

As per using path-mapping transformer, it looks like it should be relatively easy to do though. Will give it a try, thank you.

ahnpnl commented 3 years ago

ye the downside of moduleNameMapper is developers need to create an "ultimate" RegEx pattern to capture all scenarios which are not too ideal.

About custom Jest resolver, you can check https://github.com/nrwl/nx/blob/master/packages/jest/plugins/resolver.ts

In general, it's about module resolution in Jest is different from the way how webpack and Angular internal do.

Maximaximum commented 3 years ago

@ahnpnl I'm currently trying to implement a custom Jest resolver, but it seems like a customer resolver won't be able to fix the issue.

Let's take a import * as Apollo from 'apollo-angular'; line as an example.

As far as I can see, a Jest resolver only deals with resolving import paths like apollo-angular to actual absolute file paths in the filesystem (like /workspaces/my-project/frontend/node_modules/apollo-angular/bundles/ngApollo.umd.js). But it has nothing to do with handling the * as Apollo part. The resolver doesn't even get the * as Apollo (or anything like { gql } or someDefaultExport) part as an argument, it has no idea about what values are actually being imported from an es6 module.

Maximaximum commented 3 years ago

And I'm not entirely sure, but it looks like the path-mapping AST transformer has nothing to do with imported values neither, it's just dealing with paths.

Looks like there has been a misunderstanding here? The issue is caused whenever a namespaced import is used, like import * as Apollo from 'apollo-angular';. The import { gql } from 'apollo-angular'; syntax does not cause any issues. So it's not about path resolution, it's about resolving the values imported by one file to the values exported by another file.

ahnpnl commented 3 years ago

It’s about module resolution happens in Jest and partially related to how ts is compiled to js with ts-jest.

When compiling, the import namespace is converted into js which Jest will read and perform module resolution to load the necessary files.

With Angular compiler, they alter AST which will modify the import namespace to the precise import file. That is not the case here when we use ts-jest which uses simple TypeScript compiler, no magic like Angular.

So, the 2 suggestions:

custom Jest resolver: this will make Jest load the correct file for import namespace. Idk how the logic should be. Basically custom resolver will tell Jest “hey this namespace import should be resolved at precise import”. Custom resolver is a generic way to handle module resolution when moduleNameMapper becomes too complex to configure.
Custom AST transformer and put to ts-jest config to do magic like Angular compiler does. This is similar to path-mapping transformer.

Ideal solution: use Angular compiler to compile all codes before running Jest. We want to go for this ofc, but will need some time to investigate how it would play well with Jest architecture.

Maximaximum commented 3 years ago

I'm sorry @ahnpnl but I still genuinely don't get it regarding a custom Jest resolver.

Considering import * as Apollo from 'apollo-angular';, the default resolver already properly resolves the apollo-angular path to /workspaces/my-project/frontend/node_modules/apollo-angular/bundles/ngApollo.umd.js path. The resolved path is correct, so there's nothing we can do in the resolver to fix the issue. Am I wrong here?

ahnpnl commented 3 years ago

If that is the case, only modify AST is the only choice left, or using the ideal solution. The resolver solution won't work all the time, especially in the case you import a compiled js like apollo-angular.

2 suggestions are just workarounds, won't fit for all scenarios.

Maximaximum commented 3 years ago

I'm trying to write an ast trasnformer that would convert namespace imports to named imports. Actually, there's a refactoring for Typescript that does exactly that: https://github.com/microsoft/TypeScript/pull/24469/files But I can't find any documentation on how to run the Typescript refactors programmatically. Any ideas?

Maximaximum commented 3 years ago

I was finally able to create a workaround that seems to solve the issue (at least for me) and lets angular+jest unit tests with namespace imports actually run: https://www.npmjs.com/package/jest-namespace-imports-transformer

I still hope that the jest-preset-angular team is going to address this issue within jest-preset-angular itself so that my workaround (which might be quite buggy) won't be needed anymore.

ahnpnl commented 3 years ago

I guess all the logic to desugar namespace syntax lies here https://github.com/Maximaximum/jest-namespace-imports-transformer/blob/main/src/transform-script.ts#L88 ? We are happy to add it as a custom AST transformer to internal codes

Maximaximum commented 3 years ago

@ahnpnl Yes, that would be great! I can't guarantee my workaround works nicely for all cases and scenarios (I'm a total newbie with regards to Typescript compiler API, jest and jest-preset-angular, so I have easily messed up something), but so far so good: it works for me.

Nielsb85 commented 2 years ago

@ahnpnl Yes, that would be great! I can't guarantee my workaround works nicely for all cases and scenarios (I'm a total newbie with regards to Typescript compiler API, jest and jest-preset-angular, so I have easily messed up something), but so far so good: it works for me.

For me , when adding the transformer:


 ● Test suite failed to run

    Cannot find module './jest-transformer'
    Require stack:
    - /client/node_modules/jest-namespace-imports-transformer/dist/index.js
    - /client/node_modules/@jest/core/node_modules/jest-util/build/requireOrImportModule.js
    - /client/node_modules/@jest/core/node_modules/jest-util/build/index.js
    - /client/node_modules/@jest/core/build/FailedTestsInteractiveMode.js
    - /client/node_modules/@jest/core/build/plugins/FailedTestsInteractive.js
    - /client/node_modules/@jest/core/build/watch.js
    - /client/node_modules/@jest/core/build/cli/index.js
    - /client/node_modules/@jest/core/build/jest.js
    - /client/node_modules/jest/node_modules/jest-cli/build/cli/index.js
    - /client/node_modules/jest/node_modules/jest-cli/bin/jest.js
    - /client/node_modules/jest/bin/jest.js

      at Object.<anonymous> (node_modules/jest-namespace-imports-transformer/dist/index.js:16:44)

I did however, migrate to angular 13

ahnpnl commented 2 years ago

@Maximaximum I just found a new workaround that you can adjust your tsconfig.spec.json to have

{
   //...
  "include": ["src/**/*.ts"]
}

at least it fixed the issue with the sample repo.

The problem I think is similar to #1199 is that: Angular doesn't support transpile ts to js in "isolated way". Angular always requires one single TypeScript Program (see https://github.com/angular/angular/issues/43165) to process all the files together while here with Jest, we split them up into multiple workers. Compilation per worker is not the same as using one single Program.

Maximaximum commented 2 years ago

@ahnpnl Any news regarding properly fixing this issue within jest-preset-angular?

DaSchTour commented 2 years ago

@Maximaximum I just found a new workaround that you can adjust your tsconfig.spec.json to have
{
   //...
  "include": ["src/**/*.ts"]
}
at least it fixed the issue with the sample repo.

The problem I think is similar to #1199 is that: Angular doesn't support transpile ts to js in "isolated way". Angular always requires one single TypeScript Program (see angular/angular#43165) to process all the files together while here with Jest, we split them up into multiple workers. Compilation per worker is not the same as using one single Program.

This doesn't seam to work with nx repos and graphql codegen 😞

bvklingeren commented 2 years ago

I am using generated code from apollo-angular like @Maximaximum and was able to fix the issue with an ngcc run and the replacement of

{
   //...
  "include": ["**/*.spec.ts", "**/*.d.ts"]
}

with

{
   //...
  "include": ["**/*.ts"]
}

in tsconfig.spec.json

Currently using jest-preset-angular@9.0.7

ibanjo commented 2 years ago

I am using generated code from apollo-angular like @Maximaximum and was able to fix the issue with an ngcc run and the replacement of
{
   //...
  "include": ["**/*.spec.ts", "**/*.d.ts"]
}
with
{
   //...
  "include": ["**/*.ts"]
}
in tsconfig.spec.json

Currently using jest-preset-angular@9.0.7

This workaround works for me, with the (possibly trivial) caveat that, if you're importing namespaced symbols from another library in your same workspace (e.g. when using Nx Workspaces), the tsconfig.spec.json to be patched is that in the consumer library (no need to touch the exporting lib as well).

devmanbr commented 1 year ago

any solution for this? the solutions found here did not work for me.

jorgevds commented 1 year ago

Hey friends, chiming in real quick to say that the change to tsconfig.spec.json:

"include": ["src/**/*.ts"]

solved 1/3 of my issue. Changing my service and spec file name (yes, really) from:

hyphenated-name.sandbox.spec.ts

to

hyphenated-name-sandbox.service.spec.ts

fixed another third of my issue.

After that, I started getting more specific Apollo errors, about "invariant" something this and "no provider for _Apollo" that. I solved those by including a defaultOptions object into my Apollo client setup that I use exclusively for unit tests, where before I only used the link and cache object keys. This is essentially a module (read: via static method) that sets the APOLLO_OPTIONS token with some fake values to be used in test.

Finally, optionally, for anyone using Nx and MSW who may be overlooking this, make sure your library's project.json test configuration has your mockServiceWorker.js file in its assets array, and that you setup your MSW server inside of your test-setup.ts file within your library's src folder.

thymikee / jest-preset-angular