fix youtube playlist content limitation - hacktoberfest

[CodeDotJS]

• If you are looking forward to work on something during the Hacktoberfest, here's an idea on which you can work -

Issue

• https://github.com/CodeDotJS/youtube-playlist/issues/11
• https://github.com/CodeDotJS/youtube-playlist/issues/2

Level - Easy Skills required - NodeJS

PRs can be sent in 3 parts

• Main code
• Tests
• Documentation

Issue Details

Fetching contents from the YouTube Playlist is easy with youtube-playlist moudle, but the API has its own limitations. Currently, we can only fetch >=100 videos using this module. But there are playlists which have more than 100 videos, and this is the point where the API becomes less handy.

On an initial load of the page, YouTube gives us 100 videos. So we need to scroll down in order to scrap more content which isn't possible to do with cheerio.

What you can do?

1 - You can use a headless browser to load the further page.

• Example - Puppeteer

2 - Scrap the names, URL, and IDs of videos and store them in an array.

• Example

  /* Array
  [ { id: 'bgU7FeiWKzc',
  name: 'Singleton Design Pattern - Beau teaches JavaScript',
  url: 'https://youtube.com/watch?v=bgU7FeiWKzc',
  duration: 291 },
  { id: '3PUVr8jFMGg',
  name: 'Observer Design Pattern - Beau teaches JavaScript',
  url: 'https://youtube.com/watch?v=3PUVr8jFMGg',
  duration: 237 },
  { id: '3pXVHRT-amw',
  name: 'Module Design Pattern - Beau teaches JavaScript',
  url: 'https://youtube.com/watch?v=3pXVHRT-amw',
  duration: 164 },
  { id: 'KOVc5o5kURE',
  name: 'Mediator Design Pattern - Beau teaches JavaScript',
  url: 'https://youtube.com/watch?v=KOVc5o5kURE',
  duration: 309 } ]
  */

  3 - Automatically export the array as JSON, CSV, or any suitable format.

NOTE

• If you work on this issue make sure you test and document the code. Moreover, don't hesitate to ask me any question. I'll be happy to help you.

• The reason for not adding this feature of scraping URL through Puppeteer to youtube-playlist is that I don't want to make that API bloated.

Happy Coding!

[Beastskyler]

Has there been any progress on this?

[CodeDotJS]

Been a little busy, will start working on it from tomorrow. Might take some time, but will finally fix it.

[Beastskyler]

Thank you so much for actually responding, I have been doing a lot of stuff with this but the limit was the only issue I could find. I am new to coding in general so this helped me out so much and saved me so much time. I was thinking of doing hours and hours of research to make my own but since I am so new, it probably wouldn't work. Again thank you for this.

[CodeDotJS]

Do you want to work on it? Let me know. I'll be happy to guide you.

[Beastskyler]

I would love to but I feel like I am too much of a beginner to.

[Beastskyler]

How is it coming along?

[CodeDotJS]

@Beastskyler, apologies. I'm finishing my silly university assignments. I kept procrastinating, and now the deadline's in a few days.

[Beastskyler]

It's all good, university is a lot more important.

[Beastskyler]

Any new updates on this?

[Beastskyler]

Any luck?

[CodeDotJS]

@Beastskyler Implementation done with the help of puppeteer would work fine for you?

[Beastskyler]

Puppeteer would be perfect.