niklasb / dryscrape

[not actively maintained] A lightweight Python library that uses Webkit to enable easy scraping of dynamic, Javascript-heavy web pages
http://dryscrape.readthedocs.io/
MIT License
533 stars 67 forks source link

InvalidResponseError due to page load failure of a network URL #46

Open vdraceil opened 8 years ago

vdraceil commented 8 years ago

session.visit('https://www.kayak.co.in/flights/BLR-DEL/2016-02-10/2016-02-24') throws InvalidResponseError because some network URL https://stags.bluekai.com/site/* failed to load

{
  "class":"InvalidResponseError",
  "message":"Unable to load URL: https://www.kayak.co.in/flights/BLR-DEL/2016-02-10/2016-02-24
because of error loading https://stags.bluekai.com/site/undefined?ret=html&phint=Product%3Dflight&
phint=Class%3DEconomy&phint=DepartureCity%3DBLR&phint=Destination%3DDEL&
phint=TravelersChildren%3D0&phint=TravelersSeniors%3D0&phint=TravelersAdults%3D1&
phint=DepartDate%3DFebruary 10 2016&phint=ReturnDate%3DFebruary 24 
2016&phint=Travelers%3D1&phint=IncludesSaturdayNight%3Dtrue&phint=LoggedIn%3Dfalse&
phint=__bk_t%3DBLR\\u00a0to\\u00a0DEL%2C 10%2F2 \\u2013 24%2F2&phint=__bk_k%3D&
phint=__bk_l%3Dhttps%3A%2F%2Fwww.kayak.co.in%2Fflights%2FBLR-
DEL%2F2016-02-10%2F2016-02-24&limit=6&bknms=ver=2.0,ua=7ae58ad3cad58b81fe8fcf6a076b854f,t=1454521436131,m=4b4e4ecaab1f1c93ab
1f1c93ab1f1c93,k=1,lang=e7425b8d215183162151831621518316,sr=800x680x24,tzo=0,hss=true,hls
=true,idb=false,addb=undefined,odb=function,cpu=4b4e4ecaab1f1c93ab1f1c93ab1f1c93,platform=1c1
7637dbf2f8edebf2f8edebf2f8ede,notrack=,plugins=4b4e4ecaab1f1c93ab1f1c93ab1f1c93,cn=e1cf5730
3394381cdd4226b23e9b0240&r=72950631: Bad Request"
}

Is there anyway in which I can ignore this and proceed with what has been loaded?