search

deedy5 / duckduckgo_search

Search for words, documents, images, videos, news, maps and text translation using the DuckDuckGo.com search engine. Downloading files and images to a local hard drive.
MIT License
932 stars 117 forks source link

RateLimitException on text() search #145

Closed keith-nisbet closed 7 months ago

keith-nisbet commented 7 months ago

When using text search I am always seeing an error. This has been working without issue until within the last hour or so.

Detail of error -

duckduckgo_search.exceptions.DuckDuckGoSearchException: _get_url() https://links.duckduckgo.com/d.js RateLimitException: _get_url() https://links.duckduckgo.com/d.js RateLimitError: resp.status_code==202

example to reproduce error -

from duckduckgo_search import DDGS

with DDGS() as ddgs: for r in ddgs.text('live free or die', region='wt-wt', safesearch='off', timelimit='y', max_results=10): print(r)

error trace -

Traceback (most recent call last): File "/Users/keithnisbet/Bubblr/ew/ew-nlp/venv/lib/python3.9/site-packages/duckduckgo_search/duckduckgo_search.py", line 48, in _get_url raise RateLimitException(f"_get_url() {url}") duckduckgo_search.exceptions.RateLimitException: _get_url() https://links.duckduckgo.com/d.js RateLimitError: resp.status_code==202 During handling of the above exception, another exception occurred: Traceback (most recent call last): File "<input>", line 4, in <module> File "/Users/keithnisbet/Bubblr/ew/ew-nlp/venv/lib/python3.9/site-packages/duckduckgo_search/duckduckgo_search.py", line 103, in text File "/Users/keithnisbet/Bubblr/ew/ew-nlp/venv/lib/python3.9/site-packages/duckduckgo_search/duckduckgo_search.py", line 154, in _text_api cache = set() File "/Users/keithnisbet/Bubblr/ew/ew-nlp/venv/lib/python3.9/site-packages/duckduckgo_search/duckduckgo_search.py", line 57, in _get_url raise HTTPException(f"_get_url() {url} HttpError: {ex}") duckduckgo_search.exceptions.DuckDuckGoSearchException: _get_url() https://links.duckduckgo.com/d.js RateLimitException: _get_url() https://links.duckduckgo.com/d.js RateLimitError: resp.status_code==202

running both local (macOS) and in docker container in Python and duckduckgo_search v 3.9.9

marcobernhardt commented 7 months ago

I am having the same issue

AyoubMarouan commented 7 months ago

Me too even with using iproyal proxy

deedy5 commented 7 months ago

fixed in v3.9.10

deedy5 commented 6 months ago

The site blocks requests with tls-fingerprint. I had to switch from httpx to curl-cffi. Now everything works as it should, hopefully for a long time.

ewagner70 commented 3 months ago

The site blocks requests with tls-fingerprint. I had to switch from httpx to curl-cffi. Now everything works as it should, hopefully for a long time.

@deedy5 : can you provide sample code for this change?

Raphaelhpze commented 2 months ago

did any figured it out? i'm using duckduckgo-search 5.1.0 and still face the issue

deedy5 commented 2 months ago

update to v5.3.0

iwo9 commented 2 months ago

I'm using v5.3.0 and still having the same issue