search

Azure / azure-sdk-tools

Tools repository leveraged by the Azure SDK team.
MIT License
109 stars 166 forks source link

[openapi-alps] Intermittent errors in webpack loading modules due to `bindings` #7669

Closed mikeharder closed 1 week ago

mikeharder commented 5 months ago

Overview

The following error occurs intermittently:

ERROR in ../../../common/temp/node_modules/.pnpm/unix-dgram@2.0.4/node_modules/unix-dgram/lib/unix_dgram.js 4:14-33
Module not found: Error: Can't resolve 'bindings' in '/mnt/vss/_work/1/s/common/temp/node_modules/.pnpm/unix-dgram@2.0.4/node_modules/unix-dgram/lib'
resolve 'bindings' in '/mnt/vss/_work/1/s/common/temp/node_modules/.pnpm/unix-dgram@2.0.4/node_modules/unix-dgram/lib'
  Parsed request is a module
  using description file: /mnt/vss/_work/1/s/common/temp/node_modules/.pnpm/unix-dgram@2.0.4/node_modules/unix-dgram/package.json (relative path: ./lib)
    resolve as module
      /mnt/vss/_work/1/s/common/temp/node_modules/.pnpm/unix-dgram@2.0.4/node_modules/unix-dgram/lib/node_modules doesn't exist or is not a directory
      /mnt/vss/_work/1/s/common/temp/node_modules/.pnpm/unix-dgram@2.0.4/node_modules/unix-dgram/node_modules doesn't exist or is not a directory
      /mnt/vss/_work/1/s/common/temp/node_modules/.pnpm/unix-dgram@2.0.4/node_modules/node_modules doesn't exist or is not a directory
      looking for modules in /mnt/vss/_work/1/s/common/temp/node_modules/.pnpm/unix-dgram@2.0.4/node_modules
        single file module
          using description file: /mnt/vss/_work/1/s/common/temp/package.json (relative path: ./node_modules/.pnpm/unix-dgram@2.0.4/node_modules/bindings)
            no extension
              /mnt/vss/_work/1/s/common/temp/node_modules/.pnpm/unix-dgram@2.0.4/node_modules/bindings doesn't exist

https://devdiv.visualstudio.com/DevDiv/_build/results?buildId=9071026&view=logs&j=d6cab82e-c36d-5e2e-17ab-85c68fa424d7&t=e9997d98-91c5-571a-4bd7-0641d3c5be5d&l=514

It may be related to one of these recent changes, but it also might not (the error is intermittent and we don't have a lot of builds to use as data):

Attempted Fixes

  1. Set webpack.resolve.symlinks=false
    • Caused persistent build failure
konrad-jamrozik commented 2 months ago

This error continues to intermittently fail our CI pipeline. Now my office desktop also fails on it and purging cache or recloning repository does not help.

I am getting:

C:\Users\kojamroz\repos\openapi-alps\private\azure-sdk-generation\azureSdkGeneration [main ≡]> rushx build

(...)

ERROR in ../../../common/temp/node_modules/.pnpm/unix-dgram@2.0.6/node_modules/unix-dgram/lib/unix_dgram.js 4:14-33
Module not found: Error: Can't resolve 'bindings' in 'C:\Users\kojamroz\repos\openapi-alps\common\temp\node_modules\.pnpm\unix-dgram@2.0.6\node_modules\unix-dgram\lib'
resolve 'bindings' in 'C:\Users\kojamroz\repos\openapi-alps\common\temp\node_modules\.pnpm\unix-dgram@2.0.6\node_modules\unix-dgram\lib'
  Parsed request is a module
  using description file: C:\Users\kojamroz\repos\openapi-alps\common\temp\node_modules\.pnpm\unix-dgram@2.0.6\node_modules\unix-dgram\package.json (relative path: ./lib)
    resolve as module
      C:\Users\kojamroz\repos\openapi-alps\common\temp\node_modules\.pnpm\unix-dgram@2.0.6\node_modules\unix-dgram\lib\node_modules doesn't exist or is not a directory
      C:\Users\kojamroz\repos\openapi-alps\common\temp\node_modules\.pnpm\unix-dgram@2.0.6\node_modules\unix-dgram\node_modules doesn't exist or is not a directory
      C:\Users\kojamroz\repos\openapi-alps\common\temp\node_modules\.pnpm\unix-dgram@2.0.6\node_modules\node_modules doesn't exist or is not a directory
      looking for modules in C:\Users\kojamroz\repos\openapi-alps\common\temp\node_modules\.pnpm\unix-dgram@2.0.6\node_modules
        single file module
          using description file: C:\Users\kojamroz\repos\openapi-alps\common\temp\package.json (relative path: ./node_modules/.pnpm/unix-dgram@2.0.6/node_modules/bindings)
            no extension
              C:\Users\kojamroz\repos\openapi-alps\common\temp\node_modules\.pnpm\unix-dgram@2.0.6\node_modules\bindings doesn't exist
            .tsx
              C:\Users\kojamroz\repos\openapi-alps\common\temp\node_modules\.pnpm\unix-dgram@2.0.6\node_modules\bindings.tsx doesn't exist
            .ts
              C:\Users\kojamroz\repos\openapi-alps\common\temp\node_modules\.pnpm\unix-dgram@2.0.6\node_modules\bindings.ts doesn't exist
            .js
              C:\Users\kojamroz\repos\openapi-alps\common\temp\node_modules\.pnpm\unix-dgram@2.0.6\node_modules\bindings.js doesn't exist
            .json
              C:\Users\kojamroz\repos\openapi-alps\common\temp\node_modules\.pnpm\unix-dgram@2.0.6\node_modules\bindings.json doesn't exist
        C:\Users\kojamroz\repos\openapi-alps\common\temp\node_modules\.pnpm\unix-dgram@2.0.6\node_modules\bindings doesn't exist
      looking for modules in C:\Users\kojamroz\repos\openapi-alps\common\temp\node_modules\.pnpm\node_modules
        single file module
          using description file: C:\Users\kojamroz\repos\openapi-alps\common\temp\package.json (relative path: ./node_modules/.pnpm/node_modules/bindings)
            no extension
              C:\Users\kojamroz\repos\openapi-alps\common\temp\node_modules\.pnpm\node_modules\bindings doesn't exist
            .tsx
              C:\Users\kojamroz\repos\openapi-alps\common\temp\node_modules\.pnpm\node_modules\bindings.tsx doesn't exist
            .ts
              C:\Users\kojamroz\repos\openapi-alps\common\temp\node_modules\.pnpm\node_modules\bindings.ts doesn't exist
            .js
              C:\Users\kojamroz\repos\openapi-alps\common\temp\node_modules\.pnpm\node_modules\bindings.js doesn't exist
            .json
              C:\Users\kojamroz\repos\openapi-alps\common\temp\node_modules\.pnpm\node_modules\bindings.json doesn't exist
        C:\Users\kojamroz\repos\openapi-alps\common\temp\node_modules\.pnpm\node_modules\bindings doesn't exist
      C:\Users\kojamroz\repos\openapi-alps\common\temp\node_modules\node_modules doesn't exist or is not a directory
      looking for modules in C:\Users\kojamroz\repos\openapi-alps\common\temp\node_modules
        single file module
          using description file: C:\Users\kojamroz\repos\openapi-alps\common\temp\package.json (relative path: ./node_modules/bindings)
            no extension
              C:\Users\kojamroz\repos\openapi-alps\common\temp\node_modules\bindings doesn't exist
            .tsx
              C:\Users\kojamroz\repos\openapi-alps\common\temp\node_modules\bindings.tsx doesn't exist
            .ts
              C:\Users\kojamroz\repos\openapi-alps\common\temp\node_modules\bindings.ts doesn't exist
            .js
              C:\Users\kojamroz\repos\openapi-alps\common\temp\node_modules\bindings.js doesn't exist
            .json
              C:\Users\kojamroz\repos\openapi-alps\common\temp\node_modules\bindings.json doesn't exist
        C:\Users\kojamroz\repos\openapi-alps\common\temp\node_modules\bindings doesn't exist
      C:\Users\kojamroz\repos\openapi-alps\common\node_modules doesn't exist or is not a directory
      C:\Users\kojamroz\repos\openapi-alps\node_modules doesn't exist or is not a directory
      C:\Users\kojamroz\repos\node_modules doesn't exist or is not a directory
      C:\Users\kojamroz\node_modules doesn't exist or is not a directory
      C:\Users\node_modules doesn't exist or is not a directory
      C:\node_modules doesn't exist or is not a directory
 @ ../../../common/temp/node_modules/.pnpm/hot-shots@8.5.2/node_modules/hot-shots/lib/transport.js 92:16-37
 @ ../../../common/temp/node_modules/.pnpm/hot-shots@8.5.2/node_modules/hot-shots/lib/statsd.js 7:24-46
 @ ../../../common/temp/node_modules/.pnpm/hot-shots@8.5.2/node_modules/hot-shots/index.js 1:0-40
 @ ../../../common/temp/node_modules/.pnpm/@azure-tools+sdk-generation-lib@1.0.5_encoding@0.1.13_eslint@8.57.0_ts-node@9.0.0/node_modules/@azure-tools/sdk-generation-lib/dist/utils/metric/MonitorClient.js 29:28-48
 @ ../../../common/temp/node_modules/.pnpm/@azure-tools+sdk-generation-lib@1.0.5_encoding@0.1.13_eslint@8.57.0_ts-node@9.0.0/node_modules/@azure-tools/sdk-generation-lib/dist/utils/metric/index.js 17:13-39
 @ ../../../common/temp/node_modules/.pnpm/@azure-tools+sdk-generation-lib@1.0.5_encoding@0.1.13_eslint@8.57.0_ts-node@9.0.0/node_modules/@azure-tools/sdk-generation-lib/dist/utils/index.js 23:13-32
 @ ../../../common/temp/node_modules/.pnpm/@azure-tools+sdk-generation-lib@1.0.5_encoding@0.1.13_eslint@8.57.0_ts-node@9.0.0/node_modules/@azure-tools/sdk-generation-lib/dist/index.js 18:13-31
 @ ./src/index.ts 32:29-71

webpack 5.76.3 compiled with 1 error and 28 warnings in 15940 ms
Error: Failed calling tsc && npx webpack.  Exit code: 1

Looks like bindings is depended on by unix-dgram which is depended on by hot-shots which is depended on by statsd in e.g. tools/sdk-generation-pipeline/packages/sdk-generation-lib/src/utils/metric/MonitorClient.ts

Relevant entry from openapi-alps\common\config\rush\pnpm-lock.yaml

  /@azure-tools/sdk-generation-lib@1.0.5(encoding@0.1.13)(eslint@8.57.0)(ts-node@9.0.0):
    resolution: {integrity: sha1-yuHaqYC5OwQdAkNa+z9Exj/ryZU=}
    dependencies:
      '@azure/event-hubs': 5.5.2
      '@azure/storage-blob': 12.17.0(encoding@0.1.13)
      '@azure/swagger-validation-common': 0.1.2
      '@octokit/auth-app': 2.11.0(encoding@0.1.13)
      '@octokit/rest': 18.0.3(encoding@0.1.13)
      ajv: 6.12.6
      class-validator: 0.14.0
      colors: 1.4.0
      convict: 6.2.4
      hot-shots: 8.5.2
      jsonc-parser: 3.2.1
      memory-fs: 0.5.0
      mongodb: 3.7.4
      node-yaml: 3.2.0(eslint@8.57.0)
      typeorm: 0.3.12(mongodb@3.7.4)(ts-node@9.0.0)
      winston: 3.7.2

@raych1 is SDK automation using statsd telemetry? We have removed that telemetry from the rest of the code as part of Spring Grove recently:

So I want to completely remove it, also from SDK automation, and thus solve this problem once and for all.

raych1 commented 2 months ago

@konrad-jamrozik , since the shift left pipeline has been deprecated, I think we can delete these codes from openapi-alps repo:

//CC: @weshaggard

mikeharder commented 2 months ago

Bindings has an additional dependent:

 /fsevents@1.2.13:
    dependencies:
      bindings: 1.5.0

  /chokidar@2.1.8:
    optionalDependencies:
      fsevents: 1.2.13

We might be able to configure rush/pnpm to not install optionalDependencies.

konrad-jamrozik commented 1 month ago

@konrad-jamrozik , since the shift left pipeline has been deprecated, I think we can delete these codes from openapi-alps repo:

  • /private/azure-api-validation
  • /private/azure-sdk-generation
  • /private/azure-swagger-validation

//CC: @weshaggard

@raych1:

konrad-jamrozik commented 1 week ago

I believe this should be fixed after I addressed bunch of things here:

Closing.