scrapinghub / splash

Lightweight, scriptable browser as a service with an HTTP API
BSD 3-Clause "New" or "Revised" License
4.06k stars 513 forks source link

HTTPS URL not rendered via proxy #1082

Open G-markier opened 3 years ago

G-markier commented 3 years ago

We are using splash hosted on Scraping hub and having a really hard time getting HTTPS URLs to render when using a proxy. It renders just fine without a proxy but times out when using one. Example of what we are facing can be seen at the link below.

@Gallaecio and @lucywang000 have you folks come across this and are there any effective solutions or workarounds any assistance would be greatly appreciated.

https://32td6rlb-splash.scrapinghub.com/info?wait=0.5&images=1&expand=1&timeout=300&url=https%3A%2F%2Fwww.amazon.com%2F&lua_source=function+main%28splash%2C+args%29%0D%0A++%0D%0A++assert%28splash%3Ago%7Bargs.url%7D%29%0D%0A++assert%28splash%3Await%280.5%29%29%0D%0A++splash%3Aselect%28%27%23glow-ingress-line2%27%29%3Amouse_click%28%29%0D%0A++assert%28splash%3Await%280.5%29%29%0D%0A++%0D%0A++splash%3Aselect%28%27%23GLUXZipUpdateInput%27%29%3Afocus%28%29%0D%0A++splash%3Asend_text%28%2720110%27%29%0D%0A++%0D%0A++assert%28splash%3Await%28math.random+%281%2C+2%29%29%29%0D%0A%0D%0A++return+%7B%0D%0A++++html+%3D+splash%3Ahtml%28%29%2C%0D%0A++++png+%3D+splash%3Apng%28%29%0D%0A++%7D%0D%0Aend&proxy=http://user:pass@myproxy.com:30000

iamumairayub commented 3 years ago

I am also facing same issue, when I use proxy with user:pass@domain:port then even a single page request via Splash does hang forever

Here is simple Lua script I used

function main(splash, args)
  assert(splash:go{'https://api.ipify.org/?format=json'})
  assert(splash:wait(0.5))
  return {
    html = splash:html(),
    png = splash:png()
  }
end