(Reopen PLEASE) Missing dependency `request` causes an error when using Google Cloud's Vision API

tferrerm commented 2 years ago

Environment details

OS: Ubuntu 20.04 x64
Node.js version: v14.2.0
npm version: 7.24.1
google-gax version: 2.28.0

Steps to reproduce

Create a serverless Node.js project
Add the serverless-webpack plugin to your serverless.yml file and use webpack with babel to package the function that will be created
Add @google-cloud/vision as a dependency
Create a Node.js Google Function or AWS Lambda that makes use of the Google Vision API (e.g. do something similar to what is shown in this SafeSearch example)
Attempt to build the lambda with webpack

Error

WARNING in ./node_modules/retry-request/index.js 74:21-39
Module not found: Error: Can't resolve 'request' in '/home/user/project/node_modules/retry-request'
 @ ./node_modules/@google-cloud/vision/node_modules/google-gax/build/src/streamingCalls/streaming.js 22:21-45
 @ ./node_modules/@google-cloud/vision/node_modules/google-gax/build/src/index.js 64:18-55
 @ ./node_modules/@google-cloud/vision/build/src/helpers.js 24:12-33
 @ ./node_modules/@google-cloud/vision/build/src/index.js 31:16-36
 @ ./src/image-service/handler/lambdaGcloudWorker.js 38:15-46

Possible solution

Add the package request as a dependency of the project here given you've included retry-request

Workaround

Add request as a dependency of the serverless project by running npm install request --save

hkdevandla commented 2 years ago

@summer-ji-eng , can you please take a look? Thanks!

summer-ji-eng commented 2 years ago

Hi @tferrerm,

Thanks for opening an issue. Quick try on @google-cloud/vision without serverless. It works for mode in grpc, fallback/webpack and rest.

Not sure if the missing dependency relate to webpack configuration, find couple similar asking (link1, link2).

Could you provide me the serverless quick start guidance, try to reproduce the error on my local. Thanks, Summer

tferrerm commented 2 years ago

Hi, this is the serverless configuration I'm using

service: ${file(./config/global.json):SERVICE_NAME}
frameworkVersion: "2"

plugins:
  - serverless-webpack
  - serverless-prune-plugin

provider:
  name: aws
  runtime: nodejs14.x
  stage: ${opt:stage, 'dev'}
  # Examples loading from a local config file. Utilizes stage flag when invoked. If no --stage was passed into invocation, use the "dev" configuration
  region: ${file(./config/${opt:stage, 'dev'}.json):REGION}
  stackName: ${file(./config/${opt:stage, 'dev'}.json):IDENTIFER_BASE}-cfStack-stackname

custom:
  webpack:
    packager: "yarn"
    includeModules:
      forceExclude:
        - aws-sdk
  somethingTopicARN:
    Fn::ImportValue: 'trust-sns-${opt:stage,self:provider.stage}-something'
  somethingRoleARN:
    Fn::ImportValue: 'trust-sns-${opt:stage,self:provider.stage}-something-Role'
  anotherthingTopicARN:
    Fn::ImportValue: 'trust-sns-${opt:stage,self:provider.stage}-anotherthing'

package:
  individually: true
  patterns:
    - '!node_modules/@aws-sdk/**'
    - '!node_modules/aws-sdk/**'
    - '!node_modules/pretty-format/**'
    - '!node_modules/typescript/**'
    - '!node_modules/@typescript-eslint/**'
    - '!node_modules/prettier/**'
    - '!node_modules/pretty-format/**'
    - '!node_modules/@babel/**'

    - '!tests/**'
    - '!sns-serverless.yml'
    - '!s3-temp-trigger-serverless.yml'
    - '!coverage/**'
    - '!.circleci/**'
    - '!README.md'

functions:
  vertexApi:
    handler: src/something-service/handler/lambdaGcloudWorker.handler
    name: trust-use1-${opt:stage, 'dev'}-lambda-vertex
    role: ${self:custom.somethingRoleARN}
    memorySize: 1024
    timeout: 900
    maximumRetryAttempts: 0
    environment:
      NODE_ENV: ${file(./config/${opt:stage, 'dev'}.json):STAGE}
      s3PreImageBucket: 
        Ref: PreProcessingBucket
    events:
      - sns: 
          arn: { "Fn::Join" : ["", ["${self:custom.somethingTopicARN}"] ]  }
          topicName: 'trust-use1-${opt:stage,self:provider.stage}-sns-something'

  anotherVertexApi:
    handler: src/another-service/handler/lambdaWorker.handler
    name: trust-use1-${opt:stage, 'dev'}-lambda-vertex-2
    role: ${self:custom.imageModerationRoleARN}
    memorySize: 1024
    timeout: 900
    maximumRetryAttempts: 0
    environment:
      NODE_ENV: ${file(./config/${opt:stage, 'dev'}.json):STAGE}
      s3PreImageBucket: 
        Ref: PreProcessingBucket
    events:
      - sns: 
          arn: { "Fn::Join" : ["", ["${self:custom.anotherTopicARN}"] ]  }
          topicName: 'trust-use1-${opt:stage,self:provider.stage}-sns-anotherthing'

# you can add CloudFormation resource templates here
resources:
  Resources:
    PreProcessingBucket:
      Type: AWS::S3::Bucket
      Properties:
        BucketName: 'trust-use1-${opt:stage,self:provider.stage}-s3'
        AccessControl: Private
        PublicAccessBlockConfiguration:
          BlockPublicAcls: true
          BlockPublicPolicy: true
          IgnorePublicAcls: true
          RestrictPublicBuckets: true        
        BucketEncryption:
          ServerSideEncryptionConfiguration:
            - ServerSideEncryptionByDefault:
                SSEAlgorithm: AES256
        LifecycleConfiguration:
          Rules:
            -
              ExpirationInDays: 1
              Status: Enabled
              Id: "Deleted in 1 Day"
              NoncurrentVersionExpirationInDays: 1

  Outputs:        

    S3ImageBucket:
      Value:
        Ref: PreProcessingBucket
      Export:
        Name: trust-s3-${opt:stage,self:provider.stage}-vertex

hkdevandla commented 2 years ago

any updates on this one?

summer-ji-eng commented 2 years ago

@tferrerm have you tried on the links I shared before? It is not gax issue, I assume that it is the serverless or webpack dependency config issue.

summer-ji-eng commented 2 years ago

Feel free to open it up if you have further question.

ChrisChiasson commented 1 year ago

@summer-ji-eng I think this should be re-opened. I don't think it is just the OP's serverless setup because I am hitting it using only Google stuff + webpack (and I don't think webpack is at fault). To me it looks like google-gax did not install the parallel dependency of retry-request known as request (as shown on the npm page for retry-request). I am hitting this when adding webpack to firebase functions, i.e. right after firebase init functions, choosing typescript, and using the built-in hello world example.

// src/index.ts
import {https, logger} from "firebase-functions/v1";

export const helloWorld = https.onRequest((request, response) => {
  logger.info("Hello logs!", {structuredData: true});
  response.send("Hello from Firebase!");
});

// webpack.config.js
const glob = require('glob');
const path = require('path');

module.exports = function () {
  const entry = {};
  glob.sync("./src/*.ts").forEach(function (file) {
    const { name } = path.parse(file);
    entry[name] = file;
  });

  return {
    entry,
    mode: 'production',
    module: {rules: [{
      test: /\.tsx?$/,
      use: 'ts-loader',
      exclude: /node_modules/
    }]},
    output: {
      clean: true,
      filename: '[name].bundle.js',
      library: '[name]',
      libraryTarget: 'umd',
      path: path.resolve(__dirname, 'dist'), // deliberately not lib for now
    },
    target: 'node'
  };
};

Commands: firebase init funcitons choose typescript project and go into the functions directory Do basic installs: npm i --save-dev webpack webpack-cli ts-loader Run npx webpack and get the following:

...
WARNING in ./node_modules/retry-request/index.js 74:21-39
Module not found: Error: Can't resolve 'request' in '/home/chrischiasson/crto/functions/node_modules/retry-request'
 @ ./node_modules/google-gax/build/src/streamingCalls/streaming.js 22:21-45
 ...

Running npm i request makes this go away.

Anerco commented 1 year ago

still an issue

bdrolet commented 1 year ago

This still seems to be an issue for any google-gax module using request which is depricated since 2020. npm request.

As of Feb 11th 2020, request is fully deprecated. No new changes are expected land. In fact, none have landed for some time.

bsleczko commented 8 months ago

Looking at this issue, I guess that we can come here every year and celebrate google-gax using deprecated dependency. This year marks 4th request deprecation anniversary 🥳

Import trace for requested module:
./node_modules/google-gax/build/src/streamingRetryRequest.js
./node_modules/google-gax/build/src/streamingCalls/streaming.js
./node_modules/google-gax/build/src/index.js
./node_modules/@google-cloud/logging/build/src/index.js
./node_modules/@google-cloud/logging-winston/build/src/common.js
./node_modules/@google-cloud/logging-winston/build/src/index.js

prvashisht commented 8 months ago

Deployment with Cloudflare pages is also impacted.

ℹ Building Nitro Server (preset: cloudflare-pages)                                                                            nitro 11:28:26 PM

[nitro 11:28:30 PM]  ERROR  Error: Cannot resolve "request" from "/Users/.../node_modules/google-gax/build/src/streamingRetryRequest.js" and externals are not allowed!

undefined

[11:28:30 PM]  ERROR  Cannot resolve "request" from "/Users/.../node_modules/google-gax/build/src/streamingRetryRequest.js" and externals are not allowed!

  at Object.resolveId (node_modules/nitropack/dist/shared/nitro.4ea992bc.mjs:1973:17)
  at async PluginDriver.hookFirstAndGetPlugin (node_modules/rollup/dist/es/shared/node-entry.js:18539:28)
  at async resolveId (node_modules/rollup/dist/es/shared/node-entry.js:17208:26)
  at async ModuleLoader.resolveId (node_modules/rollup/dist/es/shared/node-entry.js:17622:15)
  at async node_modules/@rollup/plugin-commonjs/dist/es/index.js:785:16
  at async Promise.all (index 1)
  at async node_modules/@rollup/plugin-commonjs/dist/es/index.js:777:32
  at async rewriteRequireExpressionsAndGetImportBlock (node_modules/@rollup/plugin-commonjs/dist/es/index.js:1354:28)
  at async transformCommonjs (node_modules/@rollup/plugin-commonjs/dist/es/index.js:1928:23)
  at async transform (node_modules/rollup/dist/es/shared/node-entry.js:17540:16)