DomainTools / python_api

DomainTools Official Python API
MIT License
82 stars 32 forks source link

fixes issue with proxy usage #110

Closed neok0 closed 2 years ago

neok0 commented 2 years ago

When using proxies support in the latest version an exception is thrown. As httpx requires new format of proxies keys (see https://github.com/encode/httpx/blob/aad60a4f123801e7ce0e02bc49138e7f0f9ca0a5/httpx/_utils.py#L461)

 File "/usr/local/lib/python3.8/dist-packages/domaintools/base_results.py", line 60, in _make_request
   with Client(verify=self.api.verify_ssl, proxies=self.api.extra_request_params.get('proxies'), timeout=None) as session:
 File "/usr/local/lib/python3.8/dist-packages/httpx/_client.py", line 683, in __init__
   self._mounts: typing.Dict[URLPattern, typing.Optional[BaseTransport]] = {
 File "/usr/local/lib/python3.8/dist-packages/httpx/_client.py", line 684, in <dictcomp>
   URLPattern(key): None
 File "/usr/local/lib/python3.8/dist-packages/httpx/_utils.py", line 462, in __init__
   raise ValueError(
ValueError: Proxy keys should use proper URL forms rather than plain scheme strings. Instead of "http", use "http://"

second change is required as it seems Client().get() does not support proxies keyword anymore:

  File "//usr/local/lib/python3.8/dist-packages/domaintools/base_results.py", line 75, in _get_results
    data = self._make_request()
  File "/usr/local/lib/python3.8/dist-packages/domaintools/base_results.py", line 70, in _make_request
    return session.get(url=self.url, params=self.kwargs, **self.api.extra_request_params)
TypeError: get() got an unexpected keyword argument 'proxies'

Following code snipped was used to reproduce issue and testing changes:

from domaintools import API
from domaintools.exceptions import NotFoundException

DOMAINTOOLS = {
    'api_user': '',
    'api_key': '',
}

class DomainToolsUtils:
    def __init__(self):
        self.api_user = DOMAINTOOLS['api_user']
        self.api_key = DOMAINTOOLS['api_key']
        self.api = self._get_connection()

    def _get_connection(self):
        """
        helper method to establish connection to DomainTools API.
        :return:
        """
        proxy = 'http://proxy.example.com:8000'
        return API(self.api_user, self.api_key, proxy_url=proxy)

    def iris(self, domain):
        try:
            with self.api.iris_investigate(domain) as r:
                parsed = r.response()
        except NotFoundException:
            parsed = dict()
        return parsed

if __name__ == '__main__':
    dt = DomainToolsUtils()
    dt.iris('google.com')
dnunes-domaintools commented 2 years ago

Thanks @neok0 for the bug report and proposed fix! We're going to take a slightly different approach to the fix and will get that updated soon.

neok0 commented 2 years ago

Thanks, waiting for the update.

dnunes-domaintools commented 2 years ago

Thanks for your patience, @neok0. The team had a couple things ahead in the queue, but we've fixed this is the 1.0.1 release. We appreciate your contribution!