search

joetats / youtube_search

Tool for searching for youtube videos to avoid using their heavily rate-limited API
MIT License
220 stars 64 forks source link

Fails when YouTube serves "Suggest Products" ad #21

Open zac-garby opened 4 years ago

zac-garby commented 4 years ago 
# 1
YoutubeSearch("Hard to Concentrate - Red Hot Chili Peppers", max_results=10).to_dict()

# 2
YoutubeSearch("Hard to Concentrat - Red Hot Chili Peppers", max_results=10).to_dict()

Notice any difference? (1) returns zero results, while (2) returns all ten. Searching this manually in the YouTube search box shows that the first query results in a "Suggested Products" ad from youtube (stuff like printed lyrics etc) which presumably is confusing the web scraper.

Both of these should return a full ten results since there are plenty to scrape from the page.

zac-garby commented 4 years ago

Update: It might not be just to do with the adverts. Lots of other queries which work on the website itself fail when searching through this library. Some examples:

Paix profonde - Clearlight
Ergotrip - Clearlight
(almost anything by clearlight lol)
All At Once Forever - The Faith Healers
Tangram, Set 1 - Tangerine Dream

I'm not sure what could be causing this.

troubadixx commented 4 years ago

Hi @joetats

Awesome library!! THANKS a bunch for supplying it to the public!

I can reproduce the issue mentioned above, even though I don't see any link to "suggested products" as the TO suspects in the first post.

My observations:

YoutubeSearch("Captain Future", max_results=3).to_dict() yields 0 hits.

YoutubeSearch("aptain Future", max_results=3).to_dict() yields all expected hits.

For now my workaround does exactly the above: When YoutubeSearch does not return hits, I repeat the search with the first letter of the search term removed... ... but maybe that's not the final answer? ;-)

Can you help?

Cheers, Trouba

PS: Happy to help with testing or research, just say the word!