TooTallNate / proxy-agents

Node.js HTTP Proxy Agents Monorepo
https://proxy-agents.n8.io
873 stars 228 forks source link

Is there a way to get the CONNECT response headers? #153

Closed surlyhacker closed 1 year ago

surlyhacker commented 1 year ago

This is a re-open of issue #119. I've looked at the newest code and this is still not possible.

Hi,

Is there a way to get the response headers that are sent back to the client by the proxy in response to the CONNECT method? Typically these headers return metadata associated with the proxy itself and how it may be acting. These are distinct of course from response headers to the actual proxied request issued subsequent to the CONNECT.

If you debug into the source, it appears to capture these headers here: https://github.com/TooTallNate/proxy-agents/blob/main/packages/https-proxy-agent/src/parse-proxy-response.ts#L53 And then returns them internally in a Promise here: https://github.com/TooTallNate/proxy-agents/blob/main/packages/https-proxy-agent/src/index.ts#L133 So that buffered variable holds the headers returned by the proxy server after CONNECT is issued.

Would be really nice if this was exposed via an API and/or callback event.

if the proxy itself adds headers (to responses from the target host), then those would show up and be accessible like any other response headers and pass right through this agent. But in my case I am using a proxy that does not do that, and passed some important headers only in its own CONNECT response.

Thanks

TooTallNate commented 1 year ago

Ok, sounds good. I'm thinking there could be two different APIs for this. We could do one, or both:

Option 1

Event on the req object:

const req = https.get(url, { agent });
req.on('proxyConnect', (connect) => {
  console.log(connect.statusCode);
  // 200

  console.log(connect.headers);
  // {
  //   connection: 'keep-alive'
  // }
});

This API feels more intuitive to me, but might be problematic for wrappers around the http/https modules (i.e. node-fetch) since they don't provide access to the underlying req instance.

Option 2

Event on the Agent instance:

const agent = new HttpsProxyAgent(proxyUrl);
agent.on('proxyConnect', (connect, req) => {

});

Same connect object as above. Thinking it could be useful to also pass in the req instance, though not sure what it would be useful for off the top of my head.

surlyhacker commented 1 year ago

Thanks for the quick response! Yes I happen to use node-fetch quite a bit with the proxy agent, so Option 2 would be my vote, I think. Not opposed to also doing Option 1.

TooTallNate commented 1 year ago

Released here: https://github.com/TooTallNate/proxy-agents/releases/tag/https-proxy-agent%406.1.0

surlyhacker commented 1 year ago

Thanks! You're awesome.