deedy5 / duckduckgo_search

Search for words, documents, images, videos, news, maps and text translation using the DuckDuckGo.com search engine. Downloading files and images to a local hard drive.
MIT License
932 stars 117 forks source link

HTTPStatusError: Client error '403 Forbidden' for url #98

Closed hboen closed 1 year ago

hboen commented 1 year ago

Describe the bug When i used the python library and call ddgs.news and loop through the list I get most of the time the error. Sometime it's ok...I even added sleep(2) and still get the error

HTTPStatusError: Client error '403 Forbidden' for url 'https://duckduckgo.com/news.js?l=wt-wt&o=json&noamp=1&q=ING%20and%20%27Ralph%20Hamers%27&vqd=4-167272866629043070647631275929701091116&p=-2&df=m&s=0'

Debug log Add logging.basicConfig(level=logging.DEBUG) to the beginning of your script and attach log to the issue.

Screenshots The following code I use with DDGS() as ddgs: keywords = "ING and 'Ralph Hamers'" ddgs_news_gen = ddgs.news( keywords, region="wt-wt", safesearch="Off", timelimit="m" ) for r in ddgs_news_gen: i += 1 if i > limit: break results.append(r)

Specify this information

This library is not reliable.. Most of the time it results in http 403 error...

deedy5 commented 1 year ago

This is not a library problem, but a limitation of the news api. Try to increase the waiting time between requests, or use a proxy.