search

deedy5 / duckduckgo_search

Search for words, documents, images, videos, news, maps and text translation using the DuckDuckGo.com search engine. Downloading files and images to a local hard drive.
MIT License
1.19k stars 133 forks source link

RateLimitException on text() search #145

Closed keith-nisbet closed 11 months ago

keith-nisbet commented 11 months ago

When using text search I am always seeing an error. This has been working without issue until within the last hour or so.

Detail of error -

duckduckgo_search.exceptions.DuckDuckGoSearchException: _get_url() https://links.duckduckgo.com/d.js RateLimitException: _get_url() https://links.duckduckgo.com/d.js RateLimitError: resp.status_code==202

example to reproduce error -

from duckduckgo_search import DDGS

with DDGS() as ddgs: for r in ddgs.text('live free or die', region='wt-wt', safesearch='off', timelimit='y', max_results=10): print(r)

error trace -

Traceback (most recent call last): File "/Users/keithnisbet/Bubblr/ew/ew-nlp/venv/lib/python3.9/site-packages/duckduckgo_search/duckduckgo_search.py", line 48, in _get_url raise RateLimitException(f"_get_url() {url}") duckduckgo_search.exceptions.RateLimitException: _get_url() https://links.duckduckgo.com/d.js RateLimitError: resp.status_code==202 During handling of the above exception, another exception occurred: Traceback (most recent call last): File "<input>", line 4, in <module> File "/Users/keithnisbet/Bubblr/ew/ew-nlp/venv/lib/python3.9/site-packages/duckduckgo_search/duckduckgo_search.py", line 103, in text File "/Users/keithnisbet/Bubblr/ew/ew-nlp/venv/lib/python3.9/site-packages/duckduckgo_search/duckduckgo_search.py", line 154, in _text_api cache = set() File "/Users/keithnisbet/Bubblr/ew/ew-nlp/venv/lib/python3.9/site-packages/duckduckgo_search/duckduckgo_search.py", line 57, in _get_url raise HTTPException(f"_get_url() {url} HttpError: {ex}") duckduckgo_search.exceptions.DuckDuckGoSearchException: _get_url() https://links.duckduckgo.com/d.js RateLimitException: _get_url() https://links.duckduckgo.com/d.js RateLimitError: resp.status_code==202

running both local (macOS) and in docker container in Python and duckduckgo_search v 3.9.9

marcobernhardt commented 11 months ago

I am having the same issue

AyoubMarouan commented 11 months ago

Me too even with using iproyal proxy

deedy5 commented 11 months ago

fixed in v3.9.10

deedy5 commented 11 months ago

The site blocks requests with tls-fingerprint. I had to switch from httpx to curl-cffi. Now everything works as it should, hopefully for a long time.

ewagner70 commented 8 months ago

The site blocks requests with tls-fingerprint. I had to switch from httpx to curl-cffi. Now everything works as it should, hopefully for a long time.

@deedy5 : can you provide sample code for this change?

Raphaelhpze commented 7 months ago

did any figured it out? i'm using duckduckgo-search 5.1.0 and still face the issue

deedy5 commented 7 months ago

update to v5.3.0

iwo9 commented 7 months ago

I'm using v5.3.0 and still having the same issue