Closed jonasheschl closed 1 year ago
I have also tested the ddg
function. It is a similar story. Though I could call it twice before it not returning anymore.
I think it has to do with the recent change for vqd. I tried to disable the cache here:
And then this problem was gone. I have the same issue, that i can only do a search with a specific set of keywords once. if i change the search keywords it works (or if i wait long enough)
The api on the site has changed, I'm working on the problem
Same problem here
Following
Same Here. I have app that using this images_search. As of now i have made some temp changes that can get me images, till the fix happen.
import httpx
import json
def get_images(keywords, max_results):
url = f"https://duckduckgo.com/?va=f&t=hg&q={keywords}&iax=images&ia=images"
headers = {
"accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7",
"accept-language": "en-US,en;q=0.9,hi;q=0.8",
"cache-control": "max-age=0",
"sec-ch-ua": "\"Google Chrome\";v=\"113\", \"Chromium\";v=\"113\", \"Not-A.Brand\";v=\"24\"",
"sec-ch-ua-mobile": "?0",
"sec-ch-ua-platform": "\"Windows\"",
"sec-fetch-dest": "document",
"sec-fetch-mode": "navigate",
"sec-fetch-site": "same-origin",
"sec-fetch-user": "?1",
"sec-gpc": "1",
"upgrade-insecure-requests": "1",
"cookie": "p=-2; ah=in-en; l=in-en",
"Referer": "https://duckduckgo.com/",
"Referrer-Policy": "origin"
}
with httpx.Client() as client:
response = client.get(url, headers=headers)
print("Response status code:", response.status_code)
resp = response.text
vqd_index_start = resp.index('vqd="') + 5
vqd_index_end = resp.index('"', vqd_index_start)
vqd_bytes = resp[vqd_index_start:vqd_index_end]
print(f"vqd_bytes: {vqd_bytes}")
images_url = f"https://duckduckgo.com/i.js?o=json&q={keywords}&vqd={vqd_bytes}"
with httpx.Client() as client:
response = client.get(images_url, headers=headers)
print("Response status code:", response.status_code)
# with open("F://Python//playground//temp.json", "w") as f:
# json.dump(response.json(), f)
response = response.json()
response["results"] = response["results"][:max_results]
return response
if __name__ == "__main__":
response = get_images(keywords, 10)
print(json.dumps(response, indent=4))
# for result in response["results"]:
# print(result["image"])
Same problem. Following
Update to v3.0.2
pip install -U duckduckgo_search
try this one, worked for my code after some alteration
Could reproduce this issue both locally and on a remote Kaggle notebook.
Library version tested: 2.9.5 Python versions tested: 3.10.6 (native), 3.10.10 (conda)
Running this code once returns an array of image URLs as expected. Running the script again returns an empty array. After waiting some time (~10 minutes), the script works again in Kaggle. My best guess is that ddg changed their API policy and blocks IPs upon accessing their API through scripts. This would also explain why the script fixes itself in Kaggle, as the VM might change public IP after some time.