omkarcloud / botasaurus

The All in One Framework to build Awesome Scrapers.
https://www.omkar.cloud/botasaurus/
MIT License
1.14k stars 103 forks source link

HTTP Proxy with Username Password not working in Botasaurus 4.0.14 #109

Closed raunaqss closed 1 month ago

raunaqss commented 1 month ago

Hi there, when I try to use botasaurus 4 along with an http proxy in the following format:

http://username:pwd@domain-proxy.provider.com:PORT

I am receiving the following error:

Traceback (most recent call last):
  File "/path/to/project/search_handler_bt.py", line 26, in <module>
    print(handler(event))
  File "/path/to/project/venv/lib/python3.10/site-packages/botasaurus/decorators.py", line 656, in wrapper_browser
    current_result = run_task(data_item, False, 0)
  File "/path/to/project/venv/lib/python3.10/site-packages/botasaurus/decorators.py", line 471, in run_task
    ) = create_options_and_driver_attributes_and_close_proxy(
  File "/path/to/project/venv/lib/python3.10/site-packages/botasaurus/create_driver_utils.py", line 302, in create_options_and_driver_attributes_and_close_proxy
    options = add_proxy_options(options, proxy)
  File "/path/to/project/venv/lib/python3.10/site-packages/botasaurus_proxy_authentication/__init__.py", line 30, in add_proxy_options
    new_proxy = create_proxy(proxy)
  File "/path/to/project/venv/lib/python3.10/site-packages/botasaurus_proxy_authentication/__init__.py", line 21, in create_proxy
    return getchain().anonymizeProxy(proxy_url,  timeout=300, authInfo={})
TypeError: Proxy.__call__() got an unexpected keyword argument 'authInfo'

It works fine with the 3.x.x but not with 4.x.x. I could not zip and attach error_log since it was not created. It worked fine without the proxy.

raunaqss commented 1 month ago

This error is produced only sometimes. I will add an update if I can narrow down the event that leads to its occurrence.

Chetan11-dev commented 1 month ago

Kindly run below command to resolve it.

python -m pip install bota botasaurus_api botasaurus_driver bota botasaurus-proxy-authentication botasaurus_server --upgrade