search

tasos-py / Search-Engines-Scraper

Search google, bing, yahoo, and other search engines with python
MIT License
513 stars 137 forks source link

Showing zero results on bing #30

Closed MShirazAhmad closed 3 years ago

MShirazAhmad commented 3 years ago

My Code:

from search_engines import Bing
engine = Bing()
results = engine.search("lums")
links = results.links()
print(links)
# Searching Bing                                                                 
# []

I am getting links with zero length, whatever I search.

tasos-py commented 3 years ago

Yes, I'm getting no results too. They probably made some updates in their HTML/API. Thanks for letting me know, I'll fix it as soon as possible.

tasos-py commented 3 years ago

Actually, they didn't make any updates, they just stopped supporting our User-Agent. So, all I had to do was change the FAKE_USER_AGENT string and now it works.

MShirazAhmad commented 3 years ago

Is there any way to change FAKE_USER_AGENT in the loop without changing it directly from the source code?

tasos-py commented 3 years ago

What loop do you mean? If you want to change the User-Agent per request (although I don't see why you would do that) you could probably do it by overriding the SearchEngine._get_page() method. If you want to change the User-Agent of an engine, use .set_headers(). Is this what you mean?

MShirazAhmad commented 3 years ago

Yes, that's what I was looking for. Thanks

MShirazAhmad commented 3 years ago

Each time I run, I add 1 to the number variable, which resolved the zero results issue permanently. 

number=number+1
engine.set_headers({'User-Agent':f"Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/{number}.36 (KHTML, like Gecko) Chrome/41.0.2227.0 Safari/{number}.36"})