S3 connection timeout: uncaught exception, please report this TypeError: argument #1 unsupported type undefined

[guyguy333]

Your Environment

• verdaccio version: 5.22.1
• node version [12.x.x, 14.x.x]: Using official docker image
• package manager: yarn@1
• os: linux
• platform: docker

Describe the bug

We deploy 2, sometimes 3 instances of Verdaccio on a K8S cluster to handle load. S3 storage is used with verdaccio plugin. However, randomly, we get crash and unexpected exception. It looks like it's mainly related to S3.

To Reproduce

We don't do anything special except using Verdaccio as a cache for NPM registry. It happens randomly, mainly under load.

Expected behavior

No more crash of Verdaccio. I guess if there is a S3 error, a retry could be used as well as a proper error handling. Currently, it looks like Verdaccio crash.

Screenshots, server logs, package manager log

As we can see, both instances got the issue at similar time and stack trace is related to S3 lib.

Instance 1:

Sun, 16 Apr 2023 12:40:42 GMT http-errors deprecated non-error status code; use only 4xx or 5xx status codes at ../../verdaccio/plugins/verdaccio-aws-s3-storage/node_modules/@verdaccio/commons-api/lib/index.js:113:45
info --- auth/allow_action: access granted to: undefined 
info --- allowed access for @hapi/topo 
error--- unexpected error: Connection timed out after 120000ms
Error: Connection timed out after 120000ms
    at getError (/verdaccio/plugins/verdaccio-aws-s3-storage/node_modules/@verdaccio/commons-api/lib/index.js:113:45)
    at getCode (/verdaccio/plugins/verdaccio-aws-s3-storage/node_modules/@verdaccio/commons-api/lib/index.js:151:10)
    at convertS3Error (/verdaccio/plugins/verdaccio-aws-s3-storage/lib/s3Errors.js:54:38)
    at PassThrough.<anonymous> (/verdaccio/plugins/verdaccio-aws-s3-storage/lib/s3PackageManager.js:440:50)
    at PassThrough.emit (node:events:525:35)
    at PassThrough.emit (node:domain:489:12)
    at Request.<anonymous> (/verdaccio/plugins/verdaccio-aws-s3-storage/node_modules/aws-sdk/lib/request.js:590:14)
    at Request.callListeners (/verdaccio/plugins/verdaccio-aws-s3-storage/node_modules/aws-sdk/lib/sequential_executor.js:106:20)
    at Request.emit (/verdaccio/plugins/verdaccio-aws-s3-storage/node_modules/aws-sdk/lib/sequential_executor.js:78:10)
    at Request.emit (/verdaccio/plugins/verdaccio-aws-s3-storage/node_modules/aws-sdk/lib/request.js:686:14) 
http <-- 500, user: myusername(1.2.3.4 via 4.5.6.7), req: 'GET /acorn/-/acorn-7.4.1.tgz', error: internal server error 
error--- s3: [S3PackageManager readTarball readTarballStream event] error Connection timed out after 120000ms 
http <-- 200, user: myusername(1.2.3.4 via 4.5.6.7), req: 'GET /map-stream/-/map-stream-0.1.0.tgz', bytes: 0/5052 
fatal--- uncaught exception, please report this
TypeError: argument verdaccio/verdaccio#1 unsupported type undefined
    at createError (/verdaccio/plugins/verdaccio-aws-s3-storage/node_modules/http-errors/index.js:68:13)
    at getError (/verdaccio/plugins/verdaccio-aws-s3-storage/node_modules/@verdaccio/commons-api/lib/index.js:113:45)
    at getCode (/verdaccio/plugins/verdaccio-aws-s3-storage/node_modules/@verdaccio/commons-api/lib/index.js:151:10)
    at convertS3Error (/verdaccio/plugins/verdaccio-aws-s3-storage/lib/s3Errors.js:54:38)
    at PassThrough.<anonymous> (/verdaccio/plugins/verdaccio-aws-s3-storage/lib/s3PackageManager.js:440:50)
    at PassThrough.emit (node:events:525:35)
    at PassThrough.emit (node:domain:489:12)
    at IncomingMessage.<anonymous> (/verdaccio/plugins/verdaccio-aws-s3-storage/node_modules/aws-sdk/lib/request.js:667:18)
    at IncomingMessage.emit (node:events:513:28)
    at IncomingMessage.emit (node:domain:489:12)

Instance 2:

http <-- 200, user: myusername(1.2.3.4 via 4.5.6.7), req: 'GET /arg/-/arg-5.0.2.tgz', bytes: 0/5620 
Sun, 16 Apr 2023 12:40:41 GMT http-errors deprecated non-error status code; use only 4xx or 5xx status codes at ../../verdaccio/plugins/verdaccio-aws-s3-storage/node_modules/@verdaccio/commons-api/lib/index.js:113:45
error--- unexpected error: Connection timed out after 120000ms
Error: Connection timed out after 120000ms
    at getError (/verdaccio/plugins/verdaccio-aws-s3-storage/node_modules/@verdaccio/commons-api/lib/index.js:113:45)
    at getCode (/verdaccio/plugins/verdaccio-aws-s3-storage/node_modules/@verdaccio/commons-api/lib/index.js:151:10)
    at convertS3Error (/verdaccio/plugins/verdaccio-aws-s3-storage/lib/s3Errors.js:54:38)
    at PassThrough.<anonymous> (/verdaccio/plugins/verdaccio-aws-s3-storage/lib/s3PackageManager.js:440:50)
    at PassThrough.emit (node:events:525:35)
    at PassThrough.emit (node:domain:489:12)
    at Request.<anonymous> (/verdaccio/plugins/verdaccio-aws-s3-storage/node_modules/aws-sdk/lib/request.js:590:14)
    at Request.callListeners (/verdaccio/plugins/verdaccio-aws-s3-storage/node_modules/aws-sdk/lib/sequential_executor.js:106:20)
    at Request.emit (/verdaccio/plugins/verdaccio-aws-s3-storage/node_modules/aws-sdk/lib/sequential_executor.js:78:10)
    at Request.emit (/verdaccio/plugins/verdaccio-aws-s3-storage/node_modules/aws-sdk/lib/request.js:686:14) 
error--- s3: [S3PackageManager readTarball readTarballStream event] error Connection timed out after 120000ms 
fatal--- uncaught exception, please report this
TypeError: argument verdaccio/verdaccio#1 unsupported type undefined
    at createError (/verdaccio/plugins/verdaccio-aws-s3-storage/node_modules/http-errors/index.js:68:13)
    at getError (/verdaccio/plugins/verdaccio-aws-s3-storage/node_modules/@verdaccio/commons-api/lib/index.js:113:45)
    at getCode (/verdaccio/plugins/verdaccio-aws-s3-storage/node_modules/@verdaccio/commons-api/lib/index.js:151:10)
    at convertS3Error (/verdaccio/plugins/verdaccio-aws-s3-storage/lib/s3Errors.js:54:38)
    at PassThrough.<anonymous> (/verdaccio/plugins/verdaccio-aws-s3-storage/lib/s3PackageManager.js:440:50)
    at PassThrough.emit (node:events:525:35)
    at PassThrough.emit (node:domain:489:12)
    at IncomingMessage.<anonymous> (/verdaccio/plugins/verdaccio-aws-s3-storage/node_modules/aws-sdk/lib/request.js:667:18)
    at IncomingMessage.emit (node:events:513:28)
    at IncomingMessage.emit (node:domain:489:12)

Configuration File (cat ~/.config/verdaccio/config.yaml)

uplinks:
        npmjs:
          url: https://registry.npmjs.org/
          maxage: 720m
      packages:
        '**':
          access: $authenticated
          publish: $authenticated
          proxy: npmjs
          storage: npmjs
      auth:
        htpasswd:
          file: /verdaccio/storage/htpasswd
          max_users: -1
      storage: ./storage
      plugins: /verdaccio/plugins
      max_body_size: 150mb
      store:
        aws-s3-storage:
          bucket: xxxxxx-verdaccio
          region: fr-par
          endpoint: https://s3.{region}.xxxxxx
          s3ForcePathStyle: false
      web:
        enable: false
        login: false
        pkgManagers:
          - yarn

Environment information

Running on a K8S cluster.

Debugging output

• $ NODE_DEBUG=request verdaccio display request calls (verdaccio <--> uplinks)
• $ DEBUG=verdaccio* verdaccio enable extreme verdaccio debug mode (verdaccio api)
• $ npm -ddd prints:
• $ npm config get registry prints:

Contribute to Verdaccio

• [ ] I'm willing to fix this bug 🥇

[devmanbr]

hello @guyguy333, how did you solve the problem?

I'm getting the same error since yesterday, and it came out of nowhere. in my case I just get: TypeError: argument verdaccio/verdaccio#1 unsupported type undefined

[Tombar]

Hello, we are affected by the same bug, how can we get the updated version from https://github.com/verdaccio/monorepo/pull/676

[xoapit]

The latest version does not resolve my case. {"level":60,"time":1703501213000,"pid":8,"hostname":"cfcde7e564a2","err":{"type":"Error","message":"argument #1 unsupported type undefined","stack":"TypeError: argument #1 unsupported type undefined\n at createError (/usr/local/lib/node_modules/verdaccio-aws-s3-storage/node_modules/http-errors/index.js:68:13)\n at getError (/usr/local/lib/node_modules/verdaccio-aws-s3-storage/node_modules/@verdaccio/commons-api/lib/index.js:113:45)\n at getCode (/usr/local/lib/node_modules/verdaccio-aws-s3-storage/node_modules/@verdaccio/commons-api/lib/index.js:151:10)\n at convertS3Error (/usr/local/lib/node_modules/verdaccio-aws-s3-storage/lib/s3Errors.js:43:38)\n at Response.<anonymous> (/usr/local/lib/node_modules/verdaccio-aws-s3-storage/lib/index.js:215:56)\n at Request.<anonymous> (/usr/local/lib/node_modules/verdaccio-aws-s3-storage/node_modules/aws-sdk/lib/request.js:367:18)\n at Request.callListeners (/usr/local/lib/node_modules/verdaccio-aws-s3-storage/node_modules/aws-sdk/lib/sequential_executor.js:106:20)\n at Request.emit (/usr/local/lib/node_modules/verdaccio-aws-s3-storage/node_modules/aws-sdk/lib/sequential_executor.js:78:10)\n at Request.emit (/usr/local/lib/node_modules/verdaccio-aws-s3-storage/node_modules/aws-sdk/lib/request.js:686:14)\n at Request.transition (/usr/local/lib/node_modules/verdaccio-aws-s3-storage/node_modules/aws-sdk/lib/request.js:22:10)","code":"TypeError","time":"2023-12-25T10:46:52.998Z"},"msg":"uncaught exception, please report this\nTypeError: argument #1 unsupported type undefined\n at createError (/usr/local/lib/node_modules/verdaccio-aws-s3-storage/node_modules/http-errors/index.js:68:13)\n at getError (/usr/local/lib/node_modules/verdaccio-aws-s3-storage/node_modules/@verdaccio/commons-api/lib/index.js:113:45)\n at getCode (/usr/local/lib/node_modules/verdaccio-aws-s3-storage/node_modules/@verdaccio/commons-api/lib/index.js:151:10)\n at convertS3Error (/usr/local/lib/node_modules/verdaccio-aws-s3-storage/lib/s3Errors.js:43:38)\n at Response.<anonymous> (/usr/local/lib/node_modules/verdaccio-aws-s3-storage/lib/index.js:215:56)\n at Request.<anonymous> (/usr/local/lib/node_modules/verdaccio-aws-s3-storage/node_modules/aws-sdk/lib/request.js:367:18)\n at Request.callListeners (/usr/local/lib/node_modules/verdaccio-aws-s3-storage/node_modules/aws-sdk/lib/sequential_executor.js:106:20)\n at Request.emit (/usr/local/lib/node_modules/verdaccio-aws-s3-storage/node_modules/aws-sdk/lib/sequential_executor.js:78:10)\n at Request.emit (/usr/local/lib/node_modules/verdaccio-aws-s3-storage/node_modules/aws-sdk/lib/request.js:686:14)\n at Request.transition (/usr/local/lib/node_modules/verdaccio-aws-s3-storage/node_modules/aws-sdk/lib/request.js:22:10)"} {"level":30,"time":1703501217845,"pid":9,"hostname":"cfcde7e564a2","content":"aws-s3-storage","prefix":"verdaccio","msg":"plugin successfully loaded: verdaccio-aws-s3-storage"} {"level":30,"time":1703501217914,"pid":9,"hostname":"cfcde7e564a2","file":"/verdaccio/conf/config.yaml","msg":"config file - /verdaccio/conf/config.yaml"} (node:9) NOTE: We are formalizing our plans to enter AWS SDK for JavaScript (v2) into maintenance mode in 2023.