ERR_STREAM_WRITE_AFTER_END when service is unavailable

[Gemerich]

When the gateway routes to a route that is unavailable, the gateway crashes and return ERR_STREAM_WRITE_AFTER_END

Steps to reproduce the behavior:

1. Run the gateway normally with a route that doesn't exists like localhost:8081
2. Try to access the route calling it through the gateway, for example localhost:8081/v1/test/aa
3. See error

Expected behavior
The gateway should return 'Service unavailable' error

Returned behavior
The gateway return 'Service unavailable' error but the gateway crashes with stack:

events.js:353
      throw er; // Unhandled 'error' event
      ^

Error [ERR_STREAM_WRITE_AFTER_END]: write after end
    at writeAfterEnd (_http_outgoing.js:694:15)
    at ServerResponse.end (_http_outgoing.js:815:7)
    at node_modules/fast-proxy-lite/index.js:95:19
    at node_modules/fast-proxy-lite/lib/request.js:61:18
    at node_modules/pump/index.js:75:7
    at f (node_modules/once/once.js:25:25)
    at ClientRequest.<anonymous> (/node_modules/pump/index.js:31:21)
    at ClientRequest.f (node_modules/once/once.js:25:25)
    at ClientRequest.onerror (node_modules/end-of-stream/index.js:44:12)
    at ClientRequest.emit (events.js:388:22)
Emitted 'error' event on ServerResponse instance at:
    at writeAfterEndNT (_http_outgoing.js:753:7)
    at processTicksAndRejections (internal/process/task_queues.js:83:21) {
  code: 'ERR_STREAM_WRITE_AFTER_END'
}

Additional context
The main.ts file:

/**
 * The main class
 */
import * as gateway from 'fast-gateway'
const PORT = 8080

const server = gateway({
  routes: [
    {
      prefix: '/v1/test',
      target: 'http://localhost:8081'
    }
  ]
}).start(PORT).then(() => {
  console.log(`API Gateway listening on ${PORT} port!`)
})

package.json file:

{
  "name": "test",
  "version": "1.0.0",
  "description": "test",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1",
    "build-ts": "tsc",
    "build": "npm run build-ts",
    "start": "node dist/main.js"
  },
  "keywords": [],
  "author": "",
  "license": "ISC",
  "dependencies": {
    "fast-gateway": "^3.0.0"
  },
  "devDependencies": {
    "@types/express": "^4.17.12"
  }
}

tsconfig.json file:

{
  "compilerOptions": {
    "outDir": "dist",
    "target": "es2017",
    "module": "commonjs",
    "sourceMap": true
  }
}

[jkyberneees]

Hi @Gemerich, thanks for reporting. Can you please also describe the version of Node.js you are using and the Operating System? I can't reproduce your issue as you describe it.

Thanks

[Gemerich]

Sorry for the delay! I got covid last week and was away from work. My node version is 14.17.0 and I'm running on a MacOs BigSur 11.6

[iapain]

@jkyberneees I was able to replicate provided downstream exists and it crashes during the response, hence upstream to gateway leads to above error on fast-gateway@2.9.4 andnodejs@14.16.0. IMHO it's probably because end is being called multiple times in fast-proxy@2.0.0. I assume it's also present in fast-proxy-lite.

// index.js line 90
       if (!res.sent) {
            if (err.code === 'ECONNREFUSED' || err.code === 'ERR_HTTP2_STREAM_CANCEL') {
              res.statusCode = 503
              res.end('Service Unavailable')
            } else if (err.code === 'ECONNRESET' || err.code === 'UND_ERR_HEADERS_TIMEOUT' || err.code === 'UND_ERR_BODY_TIMEOUT') {
              res.statusCode = 504
              res.end(err.message)
            } else {
              res.statusCode = 500
              res.end(err.message)
            }
          }

@Gemerich Hope you're feeling better :)

[jkyberneees]

Hi @iapain, very much appreciated. I will reproduce and push the fix ASAP.

@Gemerich Hope you're feeling better :) +1

[jkyberneees]

Hey @iapain, @Gemerich, I have to admit I am still not able to reproduce the issue. Could you please detail the steps so I can reproduce?

Thanks in advance.

[Gemerich]

I think the catch here is to call a route that doens't exists. On the example I used the route is /v1/test and you should try to call /v1/test/aa. Your gateway returns Service Unavailble normally? The service that you're pointing must not exist.

[iapain]

@Gemerich @jkyberneees TBH this happens randomly. Try bombarding gateway with multiple concurrent connections using bench marking tool.

[jkyberneees]

Hi @Gemerich and @iapain, thanks for bringing more details into this issue. I have released a minor patch in this direction inside fast-proxy-lite.

Can you please try out v3.0.2 of fast-gateway? https://github.com/BackendStack21/fast-gateway/releases/tag/v3.0.2 I am still unable to repro this issue in Node v16.13.1

Curious to see if you are still able to reproduce this issue.

Regards

[iapain]

Hi @jkyberneees

Many thanks! It's fixed in v3.0.2 :)

Cheers!

[jkyberneees]

Thanks @iapain , @Gemerich can you confirm this issue is now resolved on your side as well?