Error ProxySchemeUnknown: Not supported proxy scheme None

[imamdigmi]

Hey Pgaref! your work looks great! I am very interested in trying this package as an API Library, I just started by installing it via PIP and trying to snippet that you wrote in Readme, but I have problems about it, below is the code I use

import time
from http_request_randomizer.requests.proxy.requestProxy import RequestProxy

if __name__ == '__main__':

    start = time.time()
    req_proxy = RequestProxy()
    print("Initialization took: {0} sec".format((time.time() - start)))
    print("Size: {0}".format(len(req_proxy.get_proxy_list())))
    print("ALL = {0} ".format(list(map(lambda x: x.get_address(), req_proxy.get_proxy_list()))))

    test_url = 'http://ipv4.icanhazip.com'

    while True:
        start = time.time()
        request = req_proxy.generate_proxied_request(test_url)
        print("Proxied Request Took: {0} sec => Status: {1}".format((time.time() - start), request.__str__()))
        if request is not None:
            print("\t Response: ip={0}".format(u''.join(request.text).encode('utf-8')))
        print("Proxy List Size: {0}".format(len(req_proxy.get_proxy_list())))

        print("-> Going to sleep..")
        time.sleep(10)

And I got these error message

Traceback (most recent call last):
  File "coba.py", line 16, in <module>
    request = req_proxy.generate_proxied_request(test_url)
  File "/Users/imamdigmi/.pyenv/versions/3.7.6/envs/data-scraping/lib/python3.7/site-packages/http_request_randomizer/requests/proxy/requestProxy.py", line 112, in generate_proxied_request
    proxies={"http": self.current_proxy.get_address(), "https": self.current_proxy.get_address()})
  File "/Users/imamdigmi/.pyenv/versions/3.7.6/envs/data-scraping/lib/python3.7/site-packages/requests/api.py", line 61, in request
    return session.request(method=method, url=url, **kwargs)
  File "/Users/imamdigmi/.pyenv/versions/3.7.6/envs/data-scraping/lib/python3.7/site-packages/requests/sessions.py", line 530, in request
    resp = self.send(prep, **send_kwargs)
  File "/Users/imamdigmi/.pyenv/versions/3.7.6/envs/data-scraping/lib/python3.7/site-packages/requests/sessions.py", line 643, in send
    r = adapter.send(request, **kwargs)
  File "/Users/imamdigmi/.pyenv/versions/3.7.6/envs/data-scraping/lib/python3.7/site-packages/requests/adapters.py", line 412, in send
    conn = self.get_connection(request.url, proxies)
  File "/Users/imamdigmi/.pyenv/versions/3.7.6/envs/data-scraping/lib/python3.7/site-packages/requests/adapters.py", line 309, in get_connection
    proxy_manager = self.proxy_manager_for(proxy)
  File "/Users/imamdigmi/.pyenv/versions/3.7.6/envs/data-scraping/lib/python3.7/site-packages/requests/adapters.py", line 199, in proxy_manager_for
    **proxy_kwargs)
  File "/Users/imamdigmi/.pyenv/versions/3.7.6/envs/data-scraping/lib/python3.7/site-packages/urllib3/poolmanager.py", line 492, in proxy_from_url
    return ProxyManager(proxy_url=url, **kw)
  File "/Users/imamdigmi/.pyenv/versions/3.7.6/envs/data-scraping/lib/python3.7/site-packages/urllib3/poolmanager.py", line 429, in __init__
    raise ProxySchemeUnknown(proxy.scheme)
urllib3.exceptions.ProxySchemeUnknown: Not supported proxy scheme None

how can i solve this problem? thank you!

[1timeusername]

@imamdigmi It would seem that the problem is with the way proxies are being passed to requests.request function in requestProxy.py file. Parameter proxies in requests.request function must be a dictionary of the following form: "protocol:port": "URL of the proxy (IP works too)". Changing line from this:

proxies={"http": self.current_proxy.get_address(), "https": self.current_proxy.get_address()}

to this:

proxies={"http:{}".format(self.current_proxy.port): self.current_proxy.get_address(), "https:{}".format(self.current_proxy.port): self.current_proxy.get_address()}

seem to fix the problem. I don't know how correct this solution is though.

EDIT: above solution only hides the problem, turns out that requests not being sent through proxy. Here is the solution that use proxies:

proxies={"http": "http://"+self.current_proxy.get_address(), "https": "https://"+self.current_proxy.get_address()}

[pgaref]

Fixed as part of #60