search

roflmuffin / node-anime-scraper

Scrapes information from Gogoanime to get Anime, Episode & Video information & urls.
59 stars 17 forks source link

A simple upgrade #30

Closed andrewprivate closed 5 years ago

andrewprivate commented 5 years ago
  1. Adds released to series info (year of release)
  2. Completed source parsing so that it can select the best source from vidstreaming

Vidstreaming stores video information in a JS script tag now.

For example:

   playerInstance.setup({
       sources: [{
           file: 'https://redirector.googlevideo.com/videoplayback?id=d6fd1b1d1023a688&itag=22&source=picasa&begin=0&requiressl=yes&mm=30&mn=sn-5hnekn7d&ms=nxu&mv=u&pl=24&sc=yes&ei=axLjXJLGHsO21gLFjpvgBw&susc=ph&app=fife&mime=video/mp4&cnr=14&dur=1368.932&lmt=1556111736605959&mt=1558384196&ipbits=0&keepalive=yes&ratebypass=yes&ip=134.19.181.230&expire=1558392459&sparams=ip,ipbits,expire,id,itag,source,requiressl,mm,mn,ms,mv,pl,sc,ei,susc,app,mime,cnr,dur,lmt&signature=A0175530ACB7AF89E4336741EFAC222F4A522CE689C257639F3E4903F98121F3.B9A7BE8208DCBBF360DF34BC15FB6BB3C3FC7DCF5819C4D18E017FE6C329C2B5&key=us0',
           label: 'Auto',
           'default': 'true',
           'type': 'mp4'
       }, {
           file: 'https://redirector.googlevideo.com/videoplayback?id=d6fd1b1d1023a688&itag=18&source=picasa&begin=0&requiressl=yes&mm=30&mn=sn-5hnekn7d&ms=nxu&mv=u&pl=24&sc=yes&ei=axLjXKmrFJ3-1wLFmrSABw&susc=ph&app=fife&mime=video/mp4&dur=1368.932&lmt=1556111213755661&mt=1558384196&ipbits=0&keepalive=yes&ratebypass=yes&ip=134.19.181.230&expire=1558392459&sparams=ip,ipbits,expire,id,itag,source,requiressl,mm,mn,ms,mv,pl,sc,ei,susc,app,mime,dur,lmt&signature=AD67ABF3B50E3B06CB85291480A2A71051FF6896C388AB4FA2E145408F9DA771.408B43C400380DA8614234B8917736626887AE09C1CBAA3DE06F758947CC95D1&key=us0',
           label: '360 P',
           'type': 'mp4'
       }, {
           file: 'https://redirector.googlevideo.com/videoplayback?id=d6fd1b1d1023a688&itag=22&source=picasa&begin=0&requiressl=yes&mm=30&mn=sn-5hnekn7d&ms=nxu&mv=u&pl=24&sc=yes&ei=axLjXJLGHsO21gLFjpvgBw&susc=ph&app=fife&mime=video/mp4&cnr=14&dur=1368.932&lmt=1556111736605959&mt=1558384196&ipbits=0&keepalive=yes&ratebypass=yes&ip=134.19.181.230&expire=1558392459&sparams=ip,ipbits,expire,id,itag,source,requiressl,mm,mn,ms,mv,pl,sc,ei,susc,app,mime,cnr,dur,lmt&signature=A0175530ACB7AF89E4336741EFAC222F4A522CE689C257639F3E4903F98121F3.B9A7BE8208DCBBF360DF34BC15FB6BB3C3FC7DCF5819C4D18E017FE6C329C2B5&key=us0',
           label: '720 P',
           'type': 'mp4'
       }],
       tracks: [{
           file: "https://cache.mload.stream/images/7fcff3c0db815d91237253071b7e7472/1.vtt",
           kind: "thumbnails"
       }],
       image: "https://cache.mload.stream/images/7fcff3c0db815d91237253071b7e7472/1_cover.jpg",
   });

Previously, with #29 , I used a lazy method of just getting the first string I encounter. However, this could lead to a video source with bad quality if multiple are available like above. To solve this, I implemented a simple JSON parser using some StackOverflow code found here

Now, it compares labels to get the video with the highest quality.

Here is an example:

Anime.search('Death note').then(function (results) {
    results[1].toAnime().then((result) => {
        result.episodes[0].fetch().then((res) => {
            console.log(res)
        })
    })
})

This will be logged:

{
    "name": "EP 1",
    "url": "https://gogoanime.io/death-note-dub-episode-1",
    "videoLinks": [
        {
            "name": "Vidstreaming",
            "url": "https://redirector.googlevideo.com/videoplayback?id=d6fd1b1d1023a688&itag=22&source=picasa&begin=0&requiressl=yes&mm=30&mn=sn-5hne6n7e&ms=nxu&mv=u&pl=24&sc=yes&ei=KRnjXJ7oBYyw1wL237fQAg&susc=ph&app=fife&mime=video/mp4&cnr=14&dur=1368.932&lmt=1556111736605959&mt=1558386054&ipbits=0&keepalive=yes&ratebypass=yes&ip=134.19.181.44&expire=1558394185&sparams=ip,ipbits,expire,id,itag,source,requiressl,mm,mn,ms,mv,pl,sc,ei,susc,app,mime,cnr,dur,lmt&signature=A83C46F0ADF8DDFD5BD345616884DA13F315B97F1EC82EC649178951DBC1F8BC.5FCEE8FF259D6925457927677E9751F3FB3803002100521703AE2EE198CBB9FA&key=us0",
            "type": "source",
            "quality": "720 P",
            "sources": [
                {
                    "file": "https://redirector.googlevideo.com/videoplayback?id=d6fd1b1d1023a688&itag=22&source=picasa&begin=0&requiressl=yes&mm=30&mn=sn-5hne6n7e&ms=nxu&mv=u&pl=24&sc=yes&ei=KRnjXJ7oBYyw1wL237fQAg&susc=ph&app=fife&mime=video/mp4&cnr=14&dur=1368.932&lmt=1556111736605959&mt=1558386054&ipbits=0&keepalive=yes&ratebypass=yes&ip=134.19.181.44&expire=1558394185&sparams=ip,ipbits,expire,id,itag,source,requiressl,mm,mn,ms,mv,pl,sc,ei,susc,app,mime,cnr,dur,lmt&signature=A83C46F0ADF8DDFD5BD345616884DA13F315B97F1EC82EC649178951DBC1F8BC.5FCEE8FF259D6925457927677E9751F3FB3803002100521703AE2EE198CBB9FA&key=us0",
                    "label": "Auto",
                    "default": "true",
                    "type": "mp4"
                },
                {
                    "file": "https://redirector.googlevideo.com/videoplayback?id=d6fd1b1d1023a688&itag=18&source=picasa&begin=0&requiressl=yes&mm=30&mn=sn-5hne6n7e&ms=nxu&mv=u&pl=24&sc=yes&ei=KBnjXNiOOcf31gKn_pyAAw&susc=ph&app=fife&mime=video/mp4&dur=1368.932&lmt=1556111213755661&mt=1558386054&ipbits=0&keepalive=yes&ratebypass=yes&ip=134.19.181.44&expire=1558394184&sparams=ip,ipbits,expire,id,itag,source,requiressl,mm,mn,ms,mv,pl,sc,ei,susc,app,mime,dur,lmt&signature=D1826DE637763DABE7A21B0C6902B8931301DFFF7776FF6475EDFFACF3D2D4D0.D0955D28A7D92D484A0BE317547FF256E3A8212DDCD84F1F4559581A02A91838&key=us0",
                    "label": "360 P",
                    "type": "mp4"
                },
                {
                    "file": "https://redirector.googlevideo.com/videoplayback?id=d6fd1b1d1023a688&itag=22&source=picasa&begin=0&requiressl=yes&mm=30&mn=sn-5hne6n7e&ms=nxu&mv=u&pl=24&sc=yes&ei=KRnjXJ7oBYyw1wL237fQAg&susc=ph&app=fife&mime=video/mp4&cnr=14&dur=1368.932&lmt=1556111736605959&mt=1558386054&ipbits=0&keepalive=yes&ratebypass=yes&ip=134.19.181.44&expire=1558394185&sparams=ip,ipbits,expire,id,itag,source,requiressl,mm,mn,ms,mv,pl,sc,ei,susc,app,mime,cnr,dur,lmt&signature=A83C46F0ADF8DDFD5BD345616884DA13F315B97F1EC82EC649178951DBC1F8BC.5FCEE8FF259D6925457927677E9751F3FB3803002100521703AE2EE198CBB9FA&key=us0",
                    "label": "720 P",
                    "type": "mp4"
                }
            ]
        },
        {
            "name": "Xstreamcdn",
            "url": "https://xstreamcdn.com/v/yx93z3mzqol",
            "type": "iframe"
        },
        {
            "name": "Vidcdn",
            "url": "//vidstreaming.io/load.php?id=OTA3OTk=&title=Death+Note+%28Dub%29+Episode+1",
            "type": "iframe"
        },
        {
            "name": "Rapidvideo",
            "url": "https://www.rapidvideo.com/e/FVOA1IN8YE",
            "type": "iframe"
        },
        {
            "name": "Mp4upload",
            "url": "https://www.mp4upload.com/embed-eu0fc38nxhd2.html",
            "type": "iframe"
        },
        {
            "name": "Oload",
            "url": "https://oload.tv/embed/XWHPVglrJGE",
            "type": "iframe"
        },
        {
            "name": "Openload",
            "url": "https://openload.co/embed/B7H9IOQu_90",
            "type": "iframe"
        },
        {
            "name": "Streamango",
            "url": "https://streamango.com/embed/qssksddrmkmpbcto",
            "type": "iframe"
        },
        {
            "name": "Estream",
            "url": "https://estream.to/embed-yg3bvwogdzmo.html",
            "type": "iframe"
        },
        {
            "name": "Thevideo",
            "url": "https://vev.io/embed/qy3pw55kwjr7",
            "type": "iframe"
        },
        {
            "name": "YourUpload",
            "url": "https://www.yourupload.com/embed/tmDPLr73qIXu",
            "type": "iframe"
        }
    ]
}
andrewprivate commented 5 years ago

I will add more and then PR this.

andrewprivate commented 5 years ago

Reopened. Added better parsing (error handling + second format)

roflmuffin commented 5 years ago

Hey Andrew,

These changes look good. Its always a struggle with these video providers since they use a lot of crappy JS to obfuscate links and theres no easy way to get around it without eval (risky) or a headless browser (expensive).

This looks like a good middleground. In future I would like to make a version that is capable of parsing a whole series of these free video hosts links, but that is probably a job for another library.