search

pgaref / HTTP_Request_Randomizer

Proxying Python Requests
http://pgaref.com/blog/python-proxy/
MIT License
151 stars 59 forks source link

BS4 error from samair proxy parser #33

Closed Hessu1337 closed 7 years ago

Hessu1337 commented 7 years ago

File "/usr/lib/python3.6/site-packages/http_request_randomizer/requests/proxy/requestProxy.py", line 55, in __init__ self.proxy_list += parsers[i].parse_proxyList() File "/usr/lib/python3.6/site-packages/http_request_randomizer/requests/parsers/SamairProxyParser.py", line 42, in parse_proxyList headings = [th.get_text() for th in table.find("tr").find_all("th")] AttributeError: 'NoneType' object has no attribute 'find'

pgaref commented 7 years ago

Hello @Hessu1337

Samair proxy provider looks unreachable to me. For instance, http://samair.ru/proxy/time-01.htm page returns a 404 error - that's why the specific proxy provider is skipped when I am listing all the available proxies. Are you sure the provider is reachable?

Cheers, Panagiotis

Hessu1337 commented 7 years ago

Hi @pgaref ,

It's reachable for me, I made a pull request with bs4 findAll fix and working like a charm here.

pgaref commented 7 years ago

Hello @Hessu1337

I guess they introduced geo-restrictions. May I ask the location(country) of your parser? Thanks for the PR, please check some inline comments.

Cheers, Panagiotis

Hessu1337 commented 7 years ago

Of course, the ip location is based in France.