Search query isn't URI encoded

SuspiciousLookingOwl / scrape-yt

Simple lib to scrape information from youtube such as search results, video information, related videos, playlist information and up next video

https://www.npmjs.com/package/scrape-yt

MIT License

9 stars 3 forks source link

Search query isn't URI encoded #4

Closed KareemH closed 4 years ago

KareemH commented 4 years ago

Every time I run my server, the videoId isn't scraped at all. I even use the maxRetryCount code as well. The probability of not scraping is high now, although it was working just a few days ago. Sadly, I can't replicate what search query caused this because I am looping through an array of 220 music artists

SuspiciousLookingOwl commented 4 years ago

What version are you using? That seems to be an older version, you can upgrade to v1.0.11 (npm update scrape-yt) and try it again, i made many improvement on scraping in v1.0.11 (https://github.com/VincentJonathan/scrape-yt/issues/2#issuecomment-617866538)

KareemH commented 4 years ago

I'm using the most recent version. When I reverted to an older version, scraping seems to be fine. In total, I scrape 220 songs, but maybe around 5 songs peculiarly scrape the wrong video ID

For instance: The search query is the artists name + title name + "official music video". The video description above the youtube video is different from the actual youtube video These are just a few examples

Sorry for the excessive opening of issues! Hope you can figure it out :)

SuspiciousLookingOwl commented 4 years ago

Whoops my bad, i didn't encode URI Component on the search keyword. The & symbol on the search keyword messed the search up. Update to v1.0.12 and try again (d1b60d545492415e99a64d35c952d1a210f4ddc3).

Thanks again for reporting this. I'm going to close this, feel free to open it again if the problem still persist.