deedy5 / duckduckgo_search

Search for words, documents, images, videos, news, maps and text translation using the DuckDuckGo.com search engine. Downloading files and images to a local hard drive.
MIT License
932 stars 117 forks source link

Text search: TypeError related to _get_url() #149

Closed wellay01 closed 6 months ago

wellay01 commented 6 months ago

UPDATE: This was an error on my end. Please feel free to delete!

Thanks for this awesome package. It is so useful! I encountered a new issue this morning: I updated to version 3.9.11, and when I do a text search, I get a TypeError that seems to be related to the _get_url function.

Here is the script I'm running:

results=[]
with DDGS() as ddgs:
    for r in ddgs.text("When was George Washington born?", safesearch='moderate', timelimit='y', max_results=3):
        results.append(r)
results

And here is the full error message:

TypeError                                 Traceback (most recent call last)
File ~/anaconda3/envs/python3/lib/python3.10/site-packages/duckduckgo_search/duckduckgo_search.py:45, in DDGS._get_url(self, method, url, **kwargs)
     44 try:
---> 45     resp = self._client.request(method, url, follow_redirects=True, **kwargs)
     46     if _is_500_in_url(str(resp.url)) or resp.status_code == 403:

TypeError: Client.request() got an unexpected keyword argument 'follow_redirects'

During handling of the above exception, another exception occurred:

DuckDuckGoSearchException                 Traceback (most recent call last)
Cell In[16], line 3
      1 results=[]
      2 with DDGS() as ddgs:
----> 3     for r in ddgs.text("When was George Washington born?", safesearch='moderate', timelimit='y', max_results=3):
      4         results.append(r)
      5 results

File ~/anaconda3/envs/python3/lib/python3.10/site-packages/duckduckgo_search/duckduckgo_search.py:106, in DDGS.text(self, keywords, region, safesearch, timelimit, backend, max_results)
    103     results = self._text_lite(keywords, region, timelimit, max_results)
    105 if results:
--> 106     for i, result in enumerate(results, start=1):
    107         yield result
    108         if max_results and i >= max_results:

File ~/anaconda3/envs/python3/lib/python3.10/site-packages/duckduckgo_search/duckduckgo_search.py:134, in DDGS._text_api(self, keywords, region, safesearch, timelimit, max_results)
    119 """DuckDuckGo text search generator. Query params: https://duckduckgo.com/params
    120 
    121 Args:
   (...)
    130 
    131 """
    132 assert keywords, "keywords is mandatory"
--> 134 vqd = self._get_vqd(keywords)
    136 payload = {
    137     "q": keywords,
    138     "kl": region,
   (...)
    145     "sp": "0",
    146 }
    147 safesearch = safesearch.lower()

File ~/anaconda3/envs/python3/lib/python3.10/site-packages/duckduckgo_search/duckduckgo_search.py:64, in DDGS._get_vqd(self, keywords)
     62 def _get_vqd(self, keywords: str) -> Optional[str]:
     63     """Get vqd value for a search query."""
---> 64     resp = self._get_url("POST", "https://duckduckgo.com/", data={"q": keywords})
     65     if resp:
     66         return _extract_vqd(resp.content, keywords)

File ~/anaconda3/envs/python3/lib/python3.10/site-packages/duckduckgo_search/duckduckgo_search.py:60, in DDGS._get_url(self, method, url, **kwargs)
     58     raise HTTPException(f"_get_url() {url} HttpError: {ex}")
     59 except Exception as ex:
---> 60     raise DuckDuckGoSearchException(f"_get_url() {url} {type(ex).__name__}: {ex}")

DuckDuckGoSearchException: _get_url() https://duckduckgo.com/ TypeError: Client.request() got an unexpected keyword argument 'follow_redirects'
deedy5 commented 6 months ago

You are using a very old version of httpx (< 0.20.0). Since version 0.20.0 the allow_redirects parameter has been replaced by follow_redirects. https://github.com/encode/httpx/blob/b471f01d668b9ef730e8843531b07b1c21b74c47/CHANGELOG.md?plain=1#L234 So just update httpx package.