search

netlify / netlify-lambda

Helps building and serving lambda functions locally and in CI environments
MIT License
594 stars 117 forks source link

netlify-lambda not finding random modules #235

Closed brielov closed 4 years ago

brielov commented 4 years ago

I have a simple graphql function and when I run netlify-lambda serve it throws errors like:

Module not found: Error: Can't resolve 'aws-sdk'

So I go ahead and install that module that has nothing to do with my code and when I try again:

Module not found: Error: Can't resolve 'vue'.

I'm not using vue in any way, I actually am using graphql without any server implementation, as raw as possible.

This is the complete error stack:

ModuleNotFoundError: Module not found: Error: Can't resolve 'vue' in '/src/lambda/node_modules/vue-template-compiler'
    at /node_modules/netlify-lambda/node_modules/webpack/lib/Compilation.js:925:10
    at /node_modules/netlify-lambda/node_modules/webpack/lib/NormalModuleFactory.js:401:22
    at /node_modules/netlify-lambda/node_modules/webpack/lib/NormalModuleFactory.js:130:21
    at /node_modules/netlify-lambda/node_modules/webpack/lib/NormalModuleFactory.js:224:22
    at /node_modules/neo-async/async.js:2830:7
    at /node_modules/neo-async/async.js:6877:13
    at /node_modules/netlify-lambda/node_modules/webpack/lib/NormalModuleFactory.js:214:25
    at /node_modules/enhanced-resolve/lib/Resolver.js:213:14
    at /node_modules/enhanced-resolve/lib/Resolver.js:285:5
    at eval (eval at create (/node_modules/tapable/lib/HookCodeFactory.js:33:10), <anonymous>:15:1)
    at /node_modules/enhanced-resolve/lib/UnsafeCachePlugin.js:44:7
    at /node_modules/enhanced-resolve/lib/Resolver.js:285:5
    at eval (eval at create (/node_modules/tapable/lib/HookCodeFactory.js:33:10), <anonymous>:15:1)
    at /node_modules/enhanced-resolve/lib/Resolver.js:285:5
    at eval (eval at create (/node_modules/tapable/lib/HookCodeFactory.js:33:10), <anonymous>:27:1)
    at /node_modules/enhanced-resolve/lib/DescriptionFilePlugin.js:67:43
resolve 'vue' in '/src/lambda/node_modules/vue-template-compiler'
  Parsed request is a module
  using description file: /src/lambda/node_modules/vue-template-compiler/package.json (relative path: .)
    resolve as module
      /src/lambda/node_modules/node_modules doesn't exist or is not a directory
      /src/node_modules doesn't exist or is not a directory
      /Users/brielov/Code/node_modules doesn't exist or is not a directory
      /Users/brielov/node_modules doesn't exist or is not a directory
      /Users/node_modules doesn't exist or is not a directory
      /node_modules doesn't exist or is not a directory
      looking for modules in /src/lambda/node_modules/vue-template-compiler/node_modules
        using description file: /src/lambda/node_modules/vue-template-compiler/package.json (relative path: ./node_modules)
          using description file: /src/lambda/node_modules/vue-template-compiler/package.json (relative path: ./node_modules/vue)
            no extension
              /src/lambda/node_modules/vue-template-compiler/node_modules/vue doesn't exist
      looking for modules in /src/lambda/node_modules
        using description file: /src/lambda/package.json (relative path: ./node_modules)
          using description file: /src/lambda/package.json (relative path: ./node_modules/vue)
            no extension
              /src/lambda/node_modules/vue doesn't exist
      looking for modules in /node_modules
        using description file: /package.json (relative path: ./node_modules)
          using description file: /package.json (relative path: ./node_modules/vue)
            no extension
              /node_modules/vue doesn't exist
            .wasm
              /src/lambda/node_modules/vue-template-compiler/node_modules/vue.wasm doesn't exist
            .wasm
              /src/lambda/node_modules/vue.wasm doesn't exist
            .wasm
              /node_modules/vue.wasm doesn't exist
            .mjs
              /src/lambda/node_modules/vue-template-compiler/node_modules/vue.mjs doesn't exist
            .mjs
              /src/lambda/node_modules/vue.mjs doesn't exist
            .mjs
              /node_modules/vue.mjs doesn't exist
            .js
              /src/lambda/node_modules/vue-template-compiler/node_modules/vue.js doesn't exist
            .js
              /src/lambda/node_modules/vue.js doesn't exist
            .js
              /node_modules/vue.js doesn't exist
            .json
              /src/lambda/node_modules/vue-template-compiler/node_modules/vue.json doesn't exist
            .json
              /src/lambda/node_modules/vue.json doesn't exist
            .json
              /node_modules/vue.json doesn't exist
            .ts
              /src/lambda/node_modules/vue-template-compiler/node_modules/vue.ts doesn't exist
            .ts
              /src/lambda/node_modules/vue.ts doesn't exist
            .ts
              /node_modules/vue.ts doesn't exist
            as directory
              /src/lambda/node_modules/vue-template-compiler/node_modules/vue doesn't exist
            as directory
              /src/lambda/node_modules/vue doesn't exist
            as directory
              /node_modules/vue doesn't exist

And this is my actual handler

export const handler = async (event, ctx) => {
  const { query, variables, operationName } = JSON.parse(event.body);
  const context = await createContext(event);
  const result = await graphql(
    schema,
    query,
    null,
    context,
    variables,
    operationName,
  );
  return reply(result, context.responseHeaders);
};
erezrokah commented 4 years ago

Sorry for the late reply. Can anyone please share a repository reproducing this issue? That would make it much easier to debug. Also, please share the output of running npx envinfo --system --binaries --npmPackages netlify-lambda

Might be related: https://github.com/netlify/netlify-lambda/issues/251 https://github.com/netlify/zip-it-and-ship-it/issues/67#issuecomment-549837499

erezrokah commented 4 years ago

Closing as stale. Please comment if still relevant

jondpenton commented 3 years ago

@erezrokah I have run into this same issue when trying to create a Netlify function for a NestJS server. Here's an example repo, which uses the NestJS TypeScript Starter Repo: https://github.com/jondpenton/netlify-lambda-resolve-dependencies.

Running yarn build:lambdas gives this output:

yarn run v1.22.10
$ netlify-lambda build lambdas
netlify-lambda: Building functions
ModuleNotFoundError: Module not found: Error: Can't resolve '@nestjs/microservices' in '/root/projects/netlify-lambda-resolve-dependencies/node_modules/@nestjs/core'
    at /root/projects/netlify-lambda-resolve-dependencies/node_modules/webpack/lib/Compilation.js:925:10
    at /root/projects/netlify-lambda-resolve-dependencies/node_modules/webpack/lib/NormalModuleFactory.js:401:22
    at /root/projects/netlify-lambda-resolve-dependencies/node_modules/webpack/lib/NormalModuleFactory.js:130:21
    at /root/projects/netlify-lambda-resolve-dependencies/node_modules/webpack/lib/NormalModuleFactory.js:224:22
    at /root/projects/netlify-lambda-resolve-dependencies/node_modules/neo-async/async.js:2830:7
    at /root/projects/netlify-lambda-resolve-dependencies/node_modules/neo-async/async.js:6877:13
    at /root/projects/netlify-lambda-resolve-dependencies/node_modules/webpack/lib/NormalModuleFactory.js:214:25
    at /root/projects/netlify-lambda-resolve-dependencies/node_modules/enhanced-resolve/lib/Resolver.js:213:14
    at /root/projects/netlify-lambda-resolve-dependencies/node_modules/enhanced-resolve/lib/Resolver.js:285:5
    at eval (eval at create (/root/projects/netlify-lambda-resolve-dependencies/node_modules/tapable/lib/HookCodeFactory.js:33:10), <anonymous>:15:1)
    at /root/projects/netlify-lambda-resolve-dependencies/node_modules/enhanced-resolve/lib/UnsafeCachePlugin.js:44:7
    at /root/projects/netlify-lambda-resolve-dependencies/node_modules/enhanced-resolve/lib/Resolver.js:285:5
    at eval (eval at create (/root/projects/netlify-lambda-resolve-dependencies/node_modules/tapable/lib/HookCodeFactory.js:33:10), <anonymous>:15:1)
    at /root/projects/netlify-lambda-resolve-dependencies/node_modules/enhanced-resolve/lib/Resolver.js:285:5
    at eval (eval at create (/root/projects/netlify-lambda-resolve-dependencies/node_modules/tapable/lib/HookCodeFactory.js:33:10), <anonymous>:27:1)
    at /root/projects/netlify-lambda-resolve-dependencies/node_modules/enhanced-resolve/lib/DescriptionFilePlugin.js:67:43
resolve '@nestjs/microservices' in '/root/projects/netlify-lambda-resolve-dependencies/node_modules/@nestjs/core'
  Parsed request is a module
  using description file: /root/projects/netlify-lambda-resolve-dependencies/node_modules/@nestjs/core/package.json (relative path: .)
    resolve as module
      /root/projects/netlify-lambda-resolve-dependencies/node_modules/@nestjs/node_modules doesn't exist or is not a directory
      /root/projects/netlify-lambda-resolve-dependencies/node_modules/node_modules doesn't exist or is not a directory
      /root/projects/node_modules doesn't exist or is not a directory
      /root/node_modules doesn't exist or is not a directory
      /node_modules doesn't exist or is not a directory
      looking for modules in /root/projects/netlify-lambda-resolve-dependencies/node_modules/@nestjs/core/node_modules
        using description file: /root/projects/netlify-lambda-resolve-dependencies/node_modules/@nestjs/core/package.json (relative path: ./node_modules)
          using description file: /root/projects/netlify-lambda-resolve-dependencies/node_modules/@nestjs/core/package.json (relative path: ./node_modules/@nestjs/microservices)
            no extension
              /root/projects/netlify-lambda-resolve-dependencies/node_modules/@nestjs/core/node_modules/@nestjs/microservices doesn't exist
      looking for modules in /root/projects/netlify-lambda-resolve-dependencies/node_modules
        using description file: /root/projects/netlify-lambda-resolve-dependencies/package.json (relative path: ./node_modules)
          using description file: /root/projects/netlify-lambda-resolve-dependencies/package.json (relative path: ./node_modules/@nestjs/microservices)
            no extension
              /root/projects/netlify-lambda-resolve-dependencies/node_modules/@nestjs/microservices doesn't exist
            .wasm
              /root/projects/netlify-lambda-resolve-dependencies/node_modules/@nestjs/core/node_modules/@nestjs/microservices.wasm doesn't exist
            .wasm
              /root/projects/netlify-lambda-resolve-dependencies/node_modules/@nestjs/microservices.wasm doesn't exist
            .mjs
              /root/projects/netlify-lambda-resolve-dependencies/node_modules/@nestjs/core/node_modules/@nestjs/microservices.mjs doesn't exist
            .mjs
              /root/projects/netlify-lambda-resolve-dependencies/node_modules/@nestjs/microservices.mjs doesn't exist
            .js
              /root/projects/netlify-lambda-resolve-dependencies/node_modules/@nestjs/core/node_modules/@nestjs/microservices.js doesn't exist
            .js
              /root/projects/netlify-lambda-resolve-dependencies/node_modules/@nestjs/microservices.js doesn't exist
            .json
              /root/projects/netlify-lambda-resolve-dependencies/node_modules/@nestjs/core/node_modules/@nestjs/microservices.json doesn't exist
            .json
              /root/projects/netlify-lambda-resolve-dependencies/node_modules/@nestjs/microservices.json doesn't exist
            .ts
              /root/projects/netlify-lambda-resolve-dependencies/node_modules/@nestjs/core/node_modules/@nestjs/microservices.ts doesn't exist
            .ts
              /root/projects/netlify-lambda-resolve-dependencies/node_modules/@nestjs/microservices.ts doesn't exist
            as directory
              /root/projects/netlify-lambda-resolve-dependencies/node_modules/@nestjs/core/node_modules/@nestjs/microservices doesn't exist
            as directory
              /root/projects/netlify-lambda-resolve-dependencies/node_modules/@nestjs/microservices doesn't exist
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

Running npx envinfo --system --binaries --npmPackages netlify-lambda gives this output:


  System:
    OS: Linux 5.4 Ubuntu 20.04.2 LTS (Focal Fossa)
    CPU: (2) x64 Intel(R) Xeon(R) CPU E5-2667 v4 @ 3.20GHz
    Memory: 5.32 GB / 7.78 GB
    Container: Yes
    Shell: 5.0.17 - /bin/bash
  Binaries:
    Node: 14.16.0 - ~/.nvm/versions/node/v14.16.0/bin/node
    Yarn: 1.22.10 - ~/.nvm/versions/node/v14.16.0/bin/yarn
    npm: 7.9.0 - ~/.nvm/versions/node/v14.16.0/bin/npm
  npmPackages:
    netlify-lambda: ^2.0.3 => 2.0.3
erezrokah commented 3 years ago

Hi @jondpenton, I'm not familiar with nestjs, but shouldn't you install @nestjs/microservices in order to reference it. I don't see it in package.json: https://github.com/jondpenton/netlify-lambda-resolve-dependencies/blob/27071253411b5ba5b569d12944553160fc13b7cb/package.json#L23

jondpenton commented 3 years ago

@erezrokah @nestjs/microservices isn't referenced in any files, but is listed as a peer dependency of @nestjs/core: https://github.com/nestjs/nest/blob/3ff484d65bcf9b2d28ae11b6a7e2a603d8533d99/packages/core/package.json#L43.

If I install @nestjs/microservices to my dev dependencies, then netlify-lambda says it can't resolve @nestjs/websockets. These are both peer dependencies of @nestjs/core. It seems as if netlify-lambda requires me to install all peer dependencies of my dependencies.

What's even worse, is it gives me similar errors for the dependencies of the peer dependencies I install. So if dependency A has peer dependencies, then I have to install those dependencies, as well as the dependencies of the peer dependencies.

This process is extremely tedious, as you have to run netlify-lambda after each install. Is this a bug with netlify-lambda or is there a recommendation to remedy this?

erezrokah commented 3 years ago

Hi @jondpenton, I think you can use npm v7 and it should install all peer dependencies by default.

jondpenton commented 3 years ago

Hey @erezrokah. It looks like npm v7 does install peer dependencies, but follows peerDependenciesMeta and doesn't install optional peer dependencies. It seems that netlify-lambda isn't checking this when resolving peer dependencies, so it throws an error when it can't find an optional peer dependency.

erezrokah commented 3 years ago

Hi @jondpenton, netlify-lambda uses webpack under the hood. They way to handle it in is via a custom config and use the externals field.

jpangelle commented 3 years ago

I am having this same issue after installing @google-cloud/storage and running netlify-lambda serve lambda:

WARNING in ../node_modules/hash-stream-validation/index.js
Module not found: Error: Can't resolve 'fast-crc32c' in 'node_modules/hash-stream-validation'
 @ ../node_modules/hash-stream-validation/index.js
 @ ../node_modules/@google-cloud/storage/build/src/file.js
 @ ../node_modules/@google-cloud/storage/build/src/index.js
 @ ./createReferralCode.js

WARNING in ../node_modules/retry-request/index.js
Module not found: Error: Can't resolve 'request' in 'node_modules/retry-request'
 @ ../node_modules/retry-request/index.js
 @ ../node_modules/@google-cloud/common/build/src/util.js
 @ ../node_modules/@google-cloud/common/build/src/index.js
 @ ../node_modules/@google-cloud/storage/build/src/channel.js
 @ ../node_modules/@google-cloud/storage/build/src/index.js
 @ ./createReferralCode.js

If I ignore the warning and try to access my endpoint, I get an error saying:

TypeError: Expected signal to be an instanceof AbortSignal