How to import the library to create the RequestProxy() object

vnjk commented 7 years ago

Hi, as above: I have tried importing RequestProxy but with no success.

from project.http.requests.proxy.requestProxy import RequestProxy

Following the style of:

from project.http.requests.parsers.UrlParser import UrlParser

The error returned is:

Traceback (most recent call last): File "crawler_copy.py", line 18, in requests_proxy = RequestProxy() File "/usr/local/lib/python2.7/dist-packages/http_request_randomizer-0.0.3-py2.7.egg/project/http/requests/proxy/requestProxy.py", line 21, in init self.userAgent = UserAgentManager() File "/usr/local/lib/python2.7/dist-packages/http_request_randomizer-0.0.3-py2.7.egg/project/http/requests/useragent/userAgent.py", line 8, in init self.useragents = self.load_user_agents(self.agent_file) File "/usr/local/lib/python2.7/dist-packages/http_request_randomizer-0.0.3-py2.7.egg/project/http/requests/useragent/userAgent.py", line 16, in load_user_agents with open(useragentsfile, 'rb') as uaf: IOError: [Errno 2] No such file or directory: '/usr/local/lib/python2.7/dist-packages/http_request_randomizer-0.0.3->py2.7.egg/project/http/requests/useragent/../data/user_agents.txt'

How can I import the library correctly? Please advise. Thank you

vnjk commented 7 years ago

nevermind, figured out the problem. cheers

pgaref commented 7 years ago

@vnjk I fixed an issure in the setup.py file and now the easiest way would be to create a new project and import the pypi library (using requirements.txt for example) http-request-randomizer==0.0.5 After create a python file and test the code below:

import time
from http.requests.proxy.requestProxy import RequestProxy

if __name__ == '__main__':
    print "Hello"
    start = time.time()
    req_proxy = RequestProxy()
    print "Initialization took: {0} sec".format((time.time() - start))
    print "Size : ", len(req_proxy.get_proxy_list())
    print " ALL = ", req_proxy.get_proxy_list()

    test_url = 'http://icanhazip.com'

    while True:
        start = time.time()
        request = req_proxy.generate_proxied_request(test_url)
        print "Proxied Request Took: {0} sec => Status: {1}".format((time.time() - start), request.__str__())
        if request is not None:
            print "\t Response: ip={0}".format(request.text)
        print "Proxy List Size: ", len(req_proxy.get_proxy_list())

        print"-> Going to sleep.."
        time.sleep(10)

Thanks for reporting and sorry for the delay!

pgaref / HTTP_Request_Randomizer

How to import the library to create the RequestProxy() object #18