search

samuel-cavalcanti / DuckDuckGo-Image-Search-API

API para Download de imagens no https://duckduckgo.com/
The Unlicense
3 stars 1 forks source link

The Api Failed to get the imagens with https://duckduckgo.com/i.js #2

Closed gpsamt closed 1 year ago

gpsamt commented 1 year ago

Hi. When running the script it takes a long time on this line.

est_search_and_download_rice_images (test_integration.DuckDuckGoApiIntegrationSuiteCase.test_search_and_download_rice_images) ... searching images 0.0%

After a certain amount of time, this message appears.

====================================================================== ERROR: test_search_and_download_rice_images (test_integration.DuckDuckGoApiIntegrationSuiteCase.test_search_and_download_rice_images)

Traceback (most recent call last): File "C:\Users\USER\AppData\Local\Programs\Python\Python311\Lib\site-packages\urllib3\connection.py", line 174, in _new_conn conn = connection.create_connection( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\USER\AppData\Local\Programs\Python\Python311\Lib\site-packages\urllib3\util\connection.py", line 95, in create_connection raise err File "C:\Users\USER\AppData\Local\Programs\Python\Python311\Lib\site-packages\urllib3\util\connection.py", line 85, in create_connection sock.connect(sa) TimeoutError: [WinError 10060] Uma tentativa de conexão falhou porque o componente conectado não respondeu corretamente após um período de tempo ou a conexão estabelecida falhou porque o host conectado não respondeu

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "C:\Users\USER\AppData\Local\Programs\Python\Python311\Lib\site-packages\urllib3\connectionpool.py", line 703, in urlopen httplib_response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "C:\Users\USER\AppData\Local\Programs\Python\Python311\Lib\site-packages\urllib3\connectionpool.py", line 386, in _make_request self._validate_conn(conn) File "C:\Users\USER\AppData\Local\Programs\Python\Python311\Lib\site-packages\urllib3\connectionpool.py", line 1042, in _validate_conn conn.connect() File "C:\Users\USER\AppData\Local\Programs\Python\Python311\Lib\site-packages\urllib3\connection.py", line 363, in connect self.sock = conn = self._new_conn() ^^^^^^^^^^^^^^^^ File "C:\Users\USER\AppData\Local\Programs\Python\Python311\Lib\site-packages\urllib3\connection.py", line 179, in _new_conn raise ConnectTimeoutError( urllib3.exceptions.ConnectTimeoutError: (<urllib3.connection.HTTPSConnection object at 0x0000021DAE589F50>, 'Connection to duckduckgo.com timed out. (connect timeout=None)')

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "C:\Users\USER\AppData\Local\Programs\Python\Python311\Lib\site-packages\requests\adapters.py", line 489, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "C:\Users\USER\AppData\Local\Programs\Python\Python311\Lib\site-packages\urllib3\connectionpool.py", line 787, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "C:\Users\USER\AppData\Local\Programs\Python\Python311\Lib\site-packages\urllib3\util\retry.py", line 592, in increment raise MaxRetryError(_pool, url, error or ResponseError(cause)) urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='duckduckgo.com', port=443): Max retries exceeded with url: / (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x0000021DAE589F50>, 'Connection to duckduckgo.com timed out. (connect timeout=None)'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "D:\Projeto\dataset\DuckDuckGo-Image-Search-API\tests\test_integration.py", line 24, in test_search_and_download_rice_images api.search_and_download( File "D:\Projeto\dataset\DuckDuckGo-Image-Search-API\duckduckgo_images_api\duckduckgo_api.py", line 107, in search_and_download url_and_titles = self.search(keywords, max_results) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\Projeto\dataset\DuckDuckGo-Image-Search-API\duckduckgo_images_api\duckduckgo_api.py", line 56, in search self.get_token(keywords) File "D:\Projeto\dataset\DuckDuckGo-Image-Search-API\duckduckgo_images_api\duckduckgo_api.py", line 29, in get_token search_obj = self.first_search(keywords) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\Projeto\dataset\DuckDuckGo-Image-Search-API\duckduckgo_images_api\duckduckgo_api.py", line 50, in first_search res = requests.post(self.__url, data=params) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\USER\AppData\Local\Programs\Python\Python311\Lib\site-packages\requests\api.py", line 115, in post return request("post", url, data=data, json=json, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\USER\AppData\Local\Programs\Python\Python311\Lib\site-packages\requests\api.py", line 59, in request return session.request(method=method, url=url, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\USER\AppData\Local\Programs\Python\Python311\Lib\site-packages\requests\sessions.py", line 587, in request resp = self.send(prep, send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\USER\AppData\Local\Programs\Python\Python311\Lib\site-packages\requests\sessions.py", line 701, in send r = adapter.send(request, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\USER\AppData\Local\Programs\Python\Python311\Lib\site-packages\requests\adapters.py", line 553, in send raise ConnectTimeout(e, request=request) requests.exceptions.ConnectTimeout: HTTPSConnectionPool(host='duckduckgo.com', port=443): Max retries exceeded with url: / (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x0000021DAE589F50>, 'Connection to duckduckgo.com timed out. (connect timeout=None)'))

====================================================================== ERROR: test_search_home_images (test_integration.DuckDuckGoApiIntegrationSuiteCase.test_search_home_images)

Traceback (most recent call last): File "C:\Users\USER\AppData\Local\Programs\Python\Python311\Lib\site-packages\urllib3\connection.py", line 174, in _new_conn conn = connection.create_connection( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\USER\AppData\Local\Programs\Python\Python311\Lib\site-packages\urllib3\util\connection.py", line 95, in create_connection raise err File "C:\Users\USER\AppData\Local\Programs\Python\Python311\Lib\site-packages\urllib3\util\connection.py", line 85, in create_connection sock.connect(sa) TimeoutError: [WinError 10060] Uma tentativa de conexão falhou porque o componente conectado não respondeu corretamente após um período de tempo ou a conexão estabelecida falhou porque o host conectado não respondeu

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "C:\Users\USER\AppData\Local\Programs\Python\Python311\Lib\site-packages\urllib3\connectionpool.py", line 703, in urlopen httplib_response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "C:\Users\USER\AppData\Local\Programs\Python\Python311\Lib\site-packages\urllib3\connectionpool.py", line 386, in _make_request self._validate_conn(conn) File "C:\Users\USER\AppData\Local\Programs\Python\Python311\Lib\site-packages\urllib3\connectionpool.py", line 1042, in _validate_conn conn.connect() File "C:\Users\USER\AppData\Local\Programs\Python\Python311\Lib\site-packages\urllib3\connection.py", line 363, in connect self.sock = conn = self._new_conn() ^^^^^^^^^^^^^^^^ File "C:\Users\USER\AppData\Local\Programs\Python\Python311\Lib\site-packages\urllib3\connection.py", line 179, in _new_conn raise ConnectTimeoutError( urllib3.exceptions.ConnectTimeoutError: (<urllib3.connection.HTTPSConnection object at 0x0000021DAE5E6010>, 'Connection to duckduckgo.com timed out. (connect timeout=None)')

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "C:\Users\USER\AppData\Local\Programs\Python\Python311\Lib\site-packages\requests\adapters.py", line 489, in send resp = conn.urlopen( ^^^^^^^^^^^^^ File "C:\Users\USER\AppData\Local\Programs\Python\Python311\Lib\site-packages\urllib3\connectionpool.py", line 787, in urlopen retries = retries.increment( ^^^^^^^^^^^^^^^^^^ File "C:\Users\USER\AppData\Local\Programs\Python\Python311\Lib\site-packages\urllib3\util\retry.py", line 592, in increment raise MaxRetryError(_pool, url, error or ResponseError(cause)) urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='duckduckgo.com', port=443): Max retries exceeded with url: / (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x0000021DAE5E6010>, 'Connection to duckduckgo.com timed out. (connect timeout=None)'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "D:\Projeto\dataset\DuckDuckGo-Image-Search-API\tests\test_integration.py", line 13, in test_search_home_images images = api.search(keywords="home", max_results=max_results) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\Projeto\dataset\DuckDuckGo-Image-Search-API\duckduckgo_images_api\duckduckgo_api.py", line 56, in search self.get_token(keywords) File "D:\Projeto\dataset\DuckDuckGo-Image-Search-API\duckduckgo_images_api\duckduckgo_api.py", line 29, in get_token search_obj = self.first_search(keywords) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\Projeto\dataset\DuckDuckGo-Image-Search-API\duckduckgo_images_api\duckduckgo_api.py", line 50, in first_search res = requests.post(self.__url, data=params) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\USER\AppData\Local\Programs\Python\Python311\Lib\site-packages\requests\api.py", line 115, in post return request("post", url, data=data, json=json, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\USER\AppData\Local\Programs\Python\Python311\Lib\site-packages\requests\api.py", line 59, in request return session.request(method=method, url=url, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\USER\AppData\Local\Programs\Python\Python311\Lib\site-packages\requests\sessions.py", line 587, in request resp = self.send(prep, send_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\USER\AppData\Local\Programs\Python\Python311\Lib\site-packages\requests\sessions.py", line 701, in send r = adapter.send(request, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\USER\AppData\Local\Programs\Python\Python311\Lib\site-packages\requests\adapters.py", line 553, in send raise ConnectTimeout(e, request=request) requests.exceptions.ConnectTimeout: HTTPSConnectionPool(host='duckduckgo.com', port=443): Max retries exceeded with url: / (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x0000021DAE5E6010>, 'Connection to duckduckgo.com timed out. (connect timeout=None)'))

Ran 2 tests in 42.309s

FAILED (errors=2)

samuel-cavalcanti commented 1 year ago

Hi, it's not the vscode the problem. If you check the url: https://duckduckgo.com/i.js,

If this error persists, please let us know: ops@duckduckgo.com

Apparently the problem is in the duckduckgo.

I will update the lib to stopping the search when that appears

gpsamt commented 1 year ago

Samuel estou precisando baixar imagens em massa ou do google ou da Duckduckgo. Será que esse erro não é por que esses dois sites estão bloqueando o acesso a esses download via script? Também tenho testado api para o google e não tenho obtido sucesso. Vi que vc é da UFRN. Sou do IFMT-Cuiabá. Desde já agradeço pela atenção.

Att.

samuel-cavalcanti commented 1 year ago

A resposta que eu posso lhe dizer é: Não sei. Mas acredito que o problema é temporário.

Teve outra pessoa que reportou o mesmo problema #1. Naquele momento Pensei exatamente no que você pensou e decidir arquivar o repositório. Só que mais um pouco mais tarde eu testei novamente a lib e ela funcionou corretamente então eu voltei abrir o repositório.

O que posso fazer é finalizar a busca e não deixar o programa rodando infinito quando isso acontecer