search

deedy5 / duckduckgo_search

Search for words, documents, images, videos, news, maps and text translation using the DuckDuckGo.com search engine. Downloading files and images to a local hard drive.
MIT License
932 stars 117 forks source link

Image search sometimes stops working #103

Closed doris7771 closed 11 months ago

doris7771 commented 11 months ago

Describe the bug The program is encountering a httpx.RemoteProtocolError and disconnects from the server without sending a response. The program breaks when downloading a specific image and it stops downloading the rest. I encountered this while performing the image search via commands in colab. I used this command: !ddgs images -k "Audi_A3" -m 180 -s off -d

Debug log Downloading [##########----------------------------------------] 37/180 20% 00:00:44 Traceback (most recent call last): File "/usr/local/lib/python3.10/dist-packages/httpx/_transports/default.py", line 60, in map_httpcore_exceptions yield File "/usr/local/lib/python3.10/dist-packages/httpx/_transports/default.py", line 353, in handle_async_request resp = await self._pool.handle_async_request(req) File "/usr/local/lib/python3.10/dist-packages/httpcore/_async/connection_pool.py", line 262, in handle_async_request raise exc File "/usr/local/lib/python3.10/dist-packages/httpcore/_async/connection_pool.py", line 245, in handle_async_request response = await connection.handle_async_request(request) File "/usr/local/lib/python3.10/dist-packages/httpcore/_async/connection.py", line 96, in handle_async_request return await self._connection.handle_async_request(request) File "/usr/local/lib/python3.10/dist-packages/httpcore/_async/http11.py", line 121, in handle_async_request raise exc File "/usr/local/lib/python3.10/dist-packages/httpcore/_async/http11.py", line 99, in handle_async_request ) = await self._receive_response_headers(**kwargs) File "/usr/local/lib/python3.10/dist-packages/httpcore/_async/http11.py", line 164, in _receive_response_headers event = await self._receive_event(timeout=timeout) File "/usr/local/lib/python3.10/dist-packages/httpcore/_async/http11.py", line 214, in _receive_event raise RemoteProtocolError(msg) httpcore.RemoteProtocolError: Server disconnected without sending a response.

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/usr/local/bin/ddgs", line 8, in sys.exit(cli()) File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 1157, in call return self.main(args, kwargs) File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 1078, in main rv = self.invoke(ctx) File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 1719, in invoke rv.append(sub_ctx.command.invoke(sub_ctx)) File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 1434, in invoke return ctx.invoke(self.callback, ctx.params) File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 783, in invoke return __callback(args, **kwargs) File "/usr/local/lib/python3.10/dist-packages/duckduckgo_search/cli.py", line 435, in images download_results(keywords, data, images=True, proxy=proxy, threads=threads) File "/usr/local/lib/python3.10/dist-packages/duckduckgo_search/cli.py", line 164, in download_results asyncio.run(_download_results(keywords, results, images, proxy)) File "/usr/lib/python3.10/asyncio/runners.py", line 44, in run return loop.run_until_complete(main) File "/usr/lib/python3.10/asyncio/base_events.py", line 649, in run_until_complete return future.result() File "/usr/local/lib/python3.10/dist-packages/duckduckgo_search/cli.py", line 157, in _download_results await future File "/usr/lib/python3.10/asyncio/tasks.py", line 571, in _wait_for_one return f.result() # May raise f.exception(). File "/usr/local/lib/python3.10/dist-packages/duckduckgo_search/cli.py", line 105, in download_file async with client.stream("GET", url) as resp: File "/usr/lib/python3.10/contextlib.py", line 199, in aenter return await anext(self.gen) File "/usr/local/lib/python3.10/dist-packages/httpx/_client.py", line 1573, in stream response = await self.send( File "/usr/local/lib/python3.10/dist-packages/httpx/_client.py", line 1617, in send response = await self._send_handling_auth( File "/usr/local/lib/python3.10/dist-packages/httpx/_client.py", line 1645, in _send_handling_auth response = await self._send_handling_redirects( File "/usr/local/lib/python3.10/dist-packages/httpx/_client.py", line 1682, in _send_handling_redirects response = await self._send_single_request(request) File "/usr/local/lib/python3.10/dist-packages/httpx/_client.py", line 1719, in _send_single_request response = await transport.handle_async_request(request) File "/usr/local/lib/python3.10/dist-packages/httpx/_transports/default.py", line 352, in handle_async_request with map_httpcore_exceptions(): File "/usr/lib/python3.10/contextlib.py", line 153, in exit self.gen.throw(typ, value, traceback) File "/usr/local/lib/python3.10/dist-packages/httpx/_transports/default.py", line 77, in map_httpcore_exceptions raise mapped_exc(message) from exc httpx.RemoteProtocolError: Server disconnected without sending a response.

Screenshots If applicable, add screenshots to help explain your problem.

Specify this information

deedy5 commented 11 months ago

update to v3.8.4 pip install -U duckduckgo_search