Host request header not set when agent in use

ladjs / superagent

Ajax for Node.js and browsers (JS HTTP client). Maintained for @forwardemail, @ladjs, @spamscanner, @breejs, @cabinjs, and @lassjs.

https://ladjs.github.io/superagent/

MIT License

16.59k stars 1.33k forks source link

Host request header not set when agent in use #1586

Closed fluggo closed 4 years ago

fluggo commented 4 years ago

I'm using a ProxyAgent from proxy-agent to send requests from superagent through a proxy.

My code boils down to:

const request = superagent('GET', 'https://www.talosintel.com/feeds/ip-filter.blf')
   .agent(proxyAgent);

This somehow prevents superagent from sending a Host header, which makes this request fail when it redirects to an S3 bucket.

fluggo commented 4 years ago

Further debugging shows that I'm wrong: the Host header is set properly on the third redirect, to talos-intelligence-site.s3.amazonaws.com:443. The problem here appears to be that S3 rejects the :443 part of the host header, which appears to be a problem on their end, as AFAIK that's allowed. Also, I'm not convinced that this is something that superagent is even responsible for, as that header seems to be sent by the agent anyways.

Sorry to bug you!