RequestError: connect ECONNREFUSED 127.0.0.1:443

franleplant commented 3 years ago

Describe the bug

Node.js version: 14
OS & version: docker/alpine

Using this library through https://github.com/panva/node-openid-client , when making the initial request of the open id connect discovery document it throws this weird error

Actual behavior

2020-12-28T21:49:39.282Z tenx:lib-auth:http >>> Request GET https://private-oidc-provider.com//.well-known/oauth-authorization-server
2020-12-28T21:49:39.282Z tenx:lib-auth:http Headers: {
  'user-agent': 'openid-client', // the one that we use works
  accept: 'application/json',
  'accept-encoding': 'gzip, deflate, br'
}
2020-12-28T21:49:39.282Z tenx:lib-auth:http Body undefined
2020-12-28T21:49:39.283Z tenx:lib-auth:http >>> Request GET https://private-oidc-provider.com/.well-known/openid-configuration
2020-12-28T21:49:39.283Z tenx:lib-auth:http Headers: {
   'user-agent': 'openid-client',// the one that we use works
  accept: 'application/json',
  'accept-encoding': 'gzip, deflate, br'
}
2020-12-28T21:49:39.283Z tenx:lib-auth:http Body undefined
2020-12-28T21:49:39.297Z tenx:lib-auth:init Error creating costar client AggregateError:
    RequestError: connect ECONNREFUSED 127.0.0.1:443
        at ClientRequest.<anonymous> (/usr/src/app/node_modules/got/dist/source/core/index.js:957:25)
        at /usr/src/app/node_modules/dd-trace/packages/dd-trace/src/scope/base.js:68:19
        at Scope._activate (/usr/src/app/node_modules/dd-trace/packages/dd-trace/src/scope/base.js:44:14)
        at Scope.activate (/usr/src/app/node_modules/dd-trace/packages/dd-trace/src/scope/base.js:13:17)
        at ClientRequest.<anonymous> (/usr/src/app/node_modules/dd-trace/packages/dd-trace/src/scope/base.js:67:20)
        at ClientRequest.req.emit (/usr/src/app/node_modules/dd-trace/packages/datadog-plugin-http/src/client.js:93:21)
        at ClientRequest.origin.emit (/usr/src/app/node_modules/@szmarczak/http-timer/dist/source/index.js:39:20)
    RequestError: connect ECONNREFUSED 127.0.0.1:443
        at ClientRequest.<anonymous> (/usr/src/app/node_modules/got/dist/source/core/index.js:957:25)
        at /usr/src/app/node_modules/dd-trace/packages/dd-trace/src/scope/base.js:68:19
        at Scope._activate (/usr/src/app/node_modules/dd-trace/packages/dd-trace/src/scope/base.js:44:14)
        at Scope.activate (/usr/src/app/node_modules/dd-trace/packages/dd-trace/src/scope/base.js:13:17)
        at ClientRequest.<anonymous> (/usr/src/app/node_modules/dd-trace/packages/dd-trace/src/scope/base.js:67:20)
        at ClientRequest.req.emit (/usr/src/app/node_modules/dd-trace/packages/datadog-plugin-http/src/client.js:93:21)
        at ClientRequest.origin.emit (/usr/src/app/node_modules/@szmarczak/http-timer/dist/source/index.js:39:20)
    at maybeSettle (/usr/src/app/node_modules/p-some/index.js:31:11)
    at /usr/src/app/node_modules/p-some/index.js:69:23
    at processTicksAndRejections (internal/process/task_queues.js:93:5)

Expected behavior

This request to work.

This request works

running the code locally on a lot of developer machines
works via curl/wget from inside the Kube cluster node and pods

But it doesn't work via node-oidc-client and got when running side the pod / container.

Code to reproduce

working on it, but it is hard because it only fails inside the container inside the pod inside the kube cluster

Checklist

[ x ] I have read the documentation.
[ x ] I have tried my code with the latest version of Node.js and Got.

Also reported in got's repo https://github.com/sindresorhus/got/issues/1515

franleplant commented 3 years ago

Any clues to solve this will be highly appreciated

panva commented 3 years ago

Any clues to solve this will be highly appreciated

There is no bug here, you're simply asking for help diagnosing your environment / build issues. Please open a Q&A discussion instead.

https://github.com/sindresorhus/got/issues/1515 or something similar monkey patching internals is likely the cause. Or just your kubernetes networking setup.

agix commented 3 years ago

I have exactly the same issue. Still debugging.

franleplant commented 3 years ago

Check the linked “got” issue, there are further clues there. Tldr: there are libraries potentially unrelated to openid-client and got that mess up with the global native Nodejs http(s) clients causing this problems, updating or removing those problematic libs will fix the problem. It’s kinda nobody’s fault but a problem in the ecosystem

On Fri, 8 Jan 2021 at 11:36 agix notifications@github.com wrote:

I have exactly the same issue. Still debugging.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/panva/node-openid-client/issues/320#issuecomment-756787300, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAOEIJ5QM3ZVRBUUZNAFLDLSY4J7BANCNFSM4VMSHU7A .

agix commented 3 years ago

awesome... will be fun to debug :). Thanks for the tips.

franleplant commented 3 years ago

Oh it’s a blast jajajaja

On Fri, 8 Jan 2021 at 12:04 agix notifications@github.com wrote:

awesome... will be fun to debug :). Thanks for the tips.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/panva/node-openid-client/issues/320#issuecomment-756802242, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAOEIJY4ZVVG7NJ3ASCT4B3SY4NIVANCNFSM4VMSHU7A .

agix commented 3 years ago

Ok I found it's related to https://github.com/DataDog/dd-trace-js usage in our case

dnikomon commented 3 years ago

I am getting this same error running locally. Every other method seems working but Issuer.discover does not.

My code is simply

Issuer.discover(url)
    .then(iss => {
      console.log('🚀 ~ file: tokenAuth.ts ~ line 45 ~ Issuer.discover ~ iss', iss)
    })
    .catch(console.error)

RequestError: connect ECONNREFUSED 127.0.0.1:443 at ClientRequest. (node_modules/openid-client/node_modules/got/dist/source/core/index.js:953:111)

agix commented 3 years ago

So you have to check all your other libraries, remove it one by one to find out the one who messed up with got.

panva commented 3 years ago

@dnikomon happy hunting!

dnikomon commented 3 years ago

~~Nope. I'm just going to fetch .well-known endpoints manually 🙄 I don't have time for this.~~

EDIT: Thank you for explaining gravity of this issue. Will bring this up today and try to come up with a real solution rather than ignoring the issue.

panva commented 3 years ago

If you don't mind that there's a dependency in your dependency tree that is messing with node's core https request capabilities, go ahead!

agix commented 3 years ago

fetching .well-known endpoints manually

if you use got to fetch it manually (maybe other http library too) it will do the same.

franleplant commented 3 years ago

also when you do any other thing with the library such as fetching user info, refreshing the token et al should get into this error so there's no easy way around it.

Perhaps this lib could be request-library-agnostic and so if people really want to can replace got for superagent (to say an example).

You can always fork it and do that yourself.

On Wed, Jan 20, 2021 at 1:52 PM agix notifications@github.com wrote:

fetching .well-known endpoints manually

if you use got to fetch it manually (maybe other http library too) it will do the same.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/panva/node-openid-client/issues/320#issuecomment-763780498, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAOEIJ6X3NYYLQLL73PJSNTS24C4TANCNFSM4VMSHU7A .

agix commented 3 years ago

Better to track down the one messing with native https requests

panva commented 3 years ago

@franleplant the problem is not with got, just like it isn't with this library. It's that e.g. agent-base that one of your dependencies patches node https.request. Got just happens to always parse a string url to an URL object that surfaces this problem with agent-base.

proof: https://github.com/panva/jose/issues/154#issuecomment-763757453 this doesn't even use got, just the node core https method that some library chooses to mess up

panva commented 3 years ago

Libraries identified to trigger this so far (note they may be intrinsic dependencies of yours and i did not look to confirm they are the last dependency in the chain)

https://github.com/DataDog/dd-trace-js https://github.com/mailjet/mailjet-apiv3-nodejs https://github.com/TooTallNate/node-agent-base mailgun-js

franleplant commented 3 years ago

But not all libraries use the same underlying http clients, nevertheless I agree with you thats better to track down the dep thats causing the problem

On Wed, 20 Jan 2021 at 14:36 Filip Skokan notifications@github.com wrote:

Libraries identified to this so far (note they may be intrinsic dependencies of yours)

https://github.com/DataDog/dd-trace-js https://github.com/mailjet/mailjet-apiv3-nodejs https://github.com/TooTallNate/node-agent-base

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/panva/node-openid-client/issues/320#issuecomment-763814331, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAOEIJZT3UZCJYVMBXYK5K3S24ICLANCNFSM4VMSHU7A .

panva commented 3 years ago

@franleplant i can guarantee that all libs use the underlying node core https module which this unfortunately designed library (agent-base) is patching.

dnikomon commented 3 years ago

Libraries identified to trigger this so far (note they may be intrinsic dependencies of yours and i did not look to confirm they are the last dependency in the chain)

https://github.com/DataDog/dd-trace-js https://github.com/mailjet/mailjet-apiv3-nodejs https://github.com/TooTallNate/node-agent-base mailgun-js

Thank you. We have mailgun-js in dependencies. And it seems it was the only package causing problems in our case.

"mailgun-js": "^0.22.0",

panva / openid-client