deedy5 / duckduckgo_search

Search for words, documents, images, videos, news, maps and text translation using the DuckDuckGo.com search engine. Downloading files and images to a local hard drive.
MIT License
932 stars 117 forks source link

_get_url() https://duckduckgo.com KeyError: 'https' #161

Closed Anonymous-Knight closed 6 months ago

Anonymous-Knight commented 6 months ago

Before you open an issue:

  1. Make sure you have the latest version installed. Check: ddgs version. Update: pip install -U duckduckgo_search
  2. Try reinstalling the library: pip install -I duckduckgo_search
  3. Make sure the site https://duckduckgo.com is accessible in your browser
  4. Try using a proxy. The site may block ip for a while.

Describe the bug duckduckgo_search.exceptions.DuckDuckGoSearchException: _get_url() https://duckduckgo.com KeyError: 'https'

Debug log Traceback (most recent call last): File "/home/zhouzhenyu05/test/kwaipilot-tool-server/src/kuaishou/search/search.py", line 111, in results = web_search('什么是okr') File "/home/zhouzhenyu05/test/kwaipilot-tool-server/src/kuaishou/search/search.py", line 94, in web_search google_result = google_search(query) File "/home/zhouzhenyu05/test/kwaipilot-tool-server/src/kuaishou/search/search.py", line 52, in google_search results = [SearchResult(title=item['title'],url=item['href'],content=item['body']) for item in search_results] File "/home/zhouzhenyu05/test/kwaipilot-tool-server/src/kuaishou/search/search.py", line 52, in results = [SearchResult(title=item['title'],url=item['href'],content=item['body']) for item in search_results] File "/opt/conda/lib/python3.9/site-packages/duckduckgo_search/duckduckgo_search.py", line 91, in text for i, result in enumerate(results, start=1): File "/opt/conda/lib/python3.9/site-packages/duckduckgo_search/duckduckgo_search.py", line 119, in _text_api vqd = self._get_vqd(keywords) File "/opt/conda/lib/python3.9/site-packages/duckduckgo_search/duckduckgo_search.py", line 54, in _get_vqd resp = self._get_url("POST", "https://duckduckgo.com", data={"q": keywords}) File "/opt/conda/lib/python3.9/site-packages/duckduckgo_search/duckduckgo_search.py", line 50, in _get_url raise DuckDuckGoSearchException(f"_get_url() {url} {type(ex).name}: {ex}") duckduckgo_search.exceptions.DuckDuckGoSearchException: _get_url() https://duckduckgo.com KeyError: 'https'

Specify this information

Anonymous-Knight commented 6 months ago

I've tried using a proxy and the site https://duckduckgo.com/ is accessible in my browser.

deedy5 commented 6 months ago

Hi. I think there's a problem with the proxy syntax. Show me the code.

Anonymous-Knight commented 6 months ago

Hi. Here is the code i use proxy: 截屏2023-12-18 14 10 57 Here is the printed proxies: 截屏2023-12-18 14 13 42 Thanks for help.

deedy5 commented 6 months ago

What's proxy type showing?

print(proxies)
print(type(proxies))
Anonymous-Knight commented 6 months ago

It is a dict 截屏2023-12-18 14 43 43

deedy5 commented 6 months ago

The problem in proxy syntax. Since v4.0.0 duckduckgo_search uses curl-cffi.

httpx:

proxies = {
    "http://": "http://localhost:8030",
    "https://": "http://localhost:8031",
}

curl-cffi:

 proxies = {
    "http": "http://localhost:8030",
    "https": "http://localhost:8031",
}
Anonymous-Knight commented 6 months ago

Thanks for your time! Help a lot.