Samair - Githubissues

pgaref / HTTP_Request_Randomizer

Proxying Python Requests

http://pgaref.com/blog/python-proxy/

MIT License

151 stars 59 forks source link

Samair #2

Closed ieguiguren closed 8 years ago

ieguiguren commented 8 years ago

Added samari.ru to be parsed

pgaref commented 8 years ago

Thanks for the PR! I like the css parsing step but there seem to be two different styles currently in the proxy page - the first one is causing parser failure:

link rel="stylesheet" href="/styles.css" type="text/css" media="screen"
link rel="stylesheet" href="/styles/b2c37.css" type="text/css" media="screen"

PS: Some proper exception handling could help in such cases. As a step further you could also take a look at issue #4 if you are interested.

Cheers, Panagiotis

ieguiguren commented 8 years ago

I've improved the extraction of the css link so, if order or page is changed, it'll still work:

for href in soup.findAll('link'):
if '/styles/' in href.get('href'):
style = "http://www.samair.ru" + href.get('href')
break
css = requests.get(style).content.split('\n')

pgaref commented 8 years ago

Looks good, thanks!