node_modules/@nestjs/schematics/dist/lib/resource/files/ts/entities/name@singular.entity.ts breaks NPM commands

[Malak67]

Bug Report

In the last 2 days I noticed an error in my pipeline after npm install, which breaks the whole pipeline from succeeding. The error seems to be caused by a file inside node modules which is interpreted by typescript and failes.

Current behavior

I am using Gitlab pipelines to push the code to a DigitalOcean droplet. After npm install when I try to run migrations, I started to see this error: Error during migration run: backend/dist/node_modules/@nestjs/schematics/dist/lib/resource/files/ts/entities/name@singular.entity.ts:1 <% if (type === 'graphql-code-first') { %>import { ObjectType, Field, Int } from '@nestjs/graphql'; ^ SyntaxError: Unexpected token '<'

The error is replicable only on the Digital Ocean server which operates on Ubuntu 20.04. I cannot replicate it on my machine as the file is not found inside node_modules.

Input Code

The content of the file from node_modules/@nestjs/schematics/dist/lib/resource/files/ts/entities/name@singular.entity.ts:1 is:

<% if (type === 'graphql-code-first') { %>import { ObjectType, Field, Int } from '@nestjs/graphql';
@ObjectType()
export class <%= singular(classify(name)) %> {
  @Field(() => Int, { description: 'Example field (placeholder)' })
  exampleField: number;
}<% } else { %>export class <%= singular(classify(name)) %> {}<% } %>
</

Expected behavior

This file should either be removed or fixed somehow as the content inside it is clearly not typescript and that`s why it is failing: node_modules/@nestjs/schematics/dist/lib/resource/files/ts/entities/name@singular.entity.ts

Environment

Nest version: 7.0.0

For Tooling issues:

• Node version: node: 12.18.3, npm: 6.14.6
• Platform: Ubuntu 20.04.

[kamilmysliwiec]

This file should either be removed or fixed somehow as the content inside it is clearly not typescript and that`s why it is failing:

This file is a schematic template and looks as expected.

I am using Gitlab pipelines to push the code to a DigitalOcean droplet. After npm install when I try to run migrations, I started to see this error:

Is there any reason your pipeline is trying to build .ts files inside the node_modules folder? It seems that your compiler is misconfigured.

Please provide a minimum reproduction repository.

[Malak67]

Hi @kamilmysliwiec It`s good to know that the schematic is not the problem. The curious thing is that this issue started to happen now. I has worked for the last 3 months but just now we started to see the issue. I am not sure what you mean that we are building .ts files inside node_modules. In the ts.config.build file, node_modules is excluded.

I am not sure we can provide a reproduction repository. We are unable to replicate it locally...

[kamilmysliwiec]

The curious thing is that this issue started to happen now. I has worked for the last 3 months but just now we started to see the issue.

It's a new schematic.

I am not sure we can provide a reproduction repository. We are unable to replicate it locally...

You can share a minimum repo with just the configuration & scripts without any domain specific classes.

[Malak67]

I think we kind of fix it. Unfortunately to create a small repo will take me a while and these days I`m full, but I will describe our workflow and solution and for now we can close the issue.

As nest doesn`t bundle the node_modules folder inside dist, our solution is to have a pipeline where we run our unit tests, create the distribution folder and on the server, copy the dist folder along with package.json and do a npm install there. By doing this we can run migrations, seeds and so on. What we noticed is that when we do npm install on the server, inside the node_modules there are some .ts files which causes this issue. Ex:

dist$ ls node_modules/@nestjs/schematics/dist/lib/resource/files/ts
dto                          __name__.gateway.spec.ts  __name__.resolver.spec.ts
entities                     __name__.gateway.ts       __name__.resolver.ts
__name__.controller.spec.ts  __name__.graphql          __name__.service.spec.ts
__name__.controller.ts       __name__.module.ts        __name__.service.ts

Our 'fix' was to also copy package-lock.json file into the dist folder on the server, and now it works. So probably there is one dependency which has a different version than the one we have on our local environment and that is creating the .ts files inside node_modules.

I hope my explanation is clear... Don`t think that this solution is the best one and we might see this issue in the future...

Thanks for your help!

[masurceac]

@Malak67 Looks like it's an issue when TypeORM tries to load entities from node_modules folder. I had the same problem and fixed it by changing the entities property as shown below: from entities: [join(__dirname, './**/*.entity{.ts,.js}')] to entities: [join(__dirname, 'apps/backend/src/**/*.entity{.ts,.js}')].

Hope this solution matches your case too!

[Malak67]

This is actualy implemented so it`s not related to it. Also, the ormconfig was working before. This changed a few days ago. Apparently when doing npm install inside the dist folder, there are some .ts files which are found in node_modules and they are causing this. With the package-lock file we are sure that we are not pulling new dependencies. I am going to do a diff soon and see which library causes this.

[npjonath]

we also have this problems since few days when deploying in production through gitlab pipeline. It was working previously and it works fine locally, even when using npm run build && node dist/main.js

[Malak67]

Are you copying the package.json on the prod server and running npm install there? If so, also copy the package-lock.json file and see if it solves the issue..for us it worked, but unsure which dependency is doing this and why is not visible on local env.

[npjonath]

We are using webpack for building our app, in order to upload sentry source map and kubernetes deployment. So basically we can't manage to do that.

[kamilmysliwiec]

Please, fix your entities glob path to not include the node_modules folder. node_modules should never be taken into consideration unless you want to explicitly include entities published as dedicated NPM packages. In this case, you should specify paths to these entities manually (instead of letting typeorm search through all installed packages).

For example, if this:

'./**/*.entity{.ts,.js}'

is your glob path + root directory = root of your application, typeorm will search for files in the node_modules folder (and this will dramatically slow down your application's bootstrap performance). Instead, you should make sure node_modules are never included (unless, as I've mentioned above, you have to do it) by using something like:

'./dist/**/*.entity{.ts,.js}'
# or './{folder1,folder2,folder3}/**/*.entity{.ts,.js}'

[kamilmysliwiec]

I've just published a new @nestjs/schematics@7.1.2 release which dynamically updates the entity template filename. In this case, even if the wrong glob path is specified in the TypeOrm settings, the template file should be ignored.

[rafaro]

I fixed this issue with node glob.

entities: [
        __dirname + '/../!(node_modules)**/*.entity.{js,ts}',
    ],