talmobi / yt-search

97 stars 31 forks source link

pageEnd doesn't seem to work #7

Closed vhpoet closed 3 years ago

vhpoet commented 4 years ago

Having pageEnd: 1 and pageEnd: 10 return the same amount of videos.

await ytSearch({
      query: `superman`,
      pageStart: 1,
      pageEnd: 10
})

both return 19 videos for me.

P. S. thanks for this lib!

talmobi commented 4 years ago

Yes I've known about this issue and have already been working on a fix on the side for a while.

I'm also hesitating if this should be removed actually, as this library is not meant for scraping. YouTube's internal throttling limits around 1 query per second, so in order to get 10 pages each request has to be delayed by 1 second.

vhpoet commented 4 years ago

Yes I've known about this issue and have already been working on a fix on the side for a while.

I'm also hesitating if this should be removed actually, as this library is not meant for scraping. YouTube's internal throttling limits around 1 query per second, so in order to get 10 pages each request has to be delayed by 1 second.

Well not scraping really. I'm looking for specific interviews, and a lot of times they're not on the first page. Also, I don't mind waiting a second per page haha.

talmobi commented 4 years ago

@vhpoet

IIRC https://github.com/TimeForANinja/node-ytsr has working paging that you can use to go through many pages of results. You may want to try that.

talmobi commented 3 years ago

closing in favour of: https://github.com/talmobi/yt-search/issues/47