search

TooTallNate / proxy-agents

Node.js HTTP Proxy Agents Monorepo
https://proxy-agents.n8.io
873 stars 228 forks source link

Making a request with proxy results in bad request #135

Closed FC5570 closed 2 years ago

FC5570 commented 2 years ago 
import Proxy from 'https-proxy-agent';

const proxyAgent = Proxy(`http://${ip}`);
await fetch(url, {
    agent: proxyAgent,
    headers: {
        'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/102.0.5005.72 Safari/537.36'
    }
}).then(res => res.text()).then(console.log)

I'm scraping a site with the proxy, but the response is a bad request.

Here is what the response looks like: image

I have absolutely no idea why this is happening. If you want to reproduce the issue, I'm scraping https://azlyrics.com. Please let me know what is wrong.

FC5570 commented 2 years ago

I have fixed this issue. I was making a request to a site with https protocol but the proxy was http only. Using https proxies works.