rleroi / stremio-anime

Discontinued Anime add-on for Stremio
0 stars 0 forks source link

Optimizing Requests #6

Closed BoredLama closed 5 years ago

BoredLama commented 5 years ago

I noticed that this add-on does not keep a cache to optimize the number of http(s) requests it does.

Proposed logic for .defineMetaHandler and .defineCatalogHandler:

var keepData = {}
...
addon.defineMetaHandler((args, cb) => {
    var apiParams = 'action=load_episodes&movie_id='+args.id.split(':')[1];
    if (keepData[apiParams]) {
        cb(null, { meta: keepData[apiParams] });
    } else {
            axios.put(apiUrl, apiParams, apiOptions)
            ... // do logic
            // save data before responding:
            keepData[apiParams] = dataset
            // set timeout for data to 1 day (or more, your choice)
            setTimeout(() => { delete keepData[apiParams] }, 86400000)
            // respond with data
            cb(null, { meta: dataset })
    }
})
// same logic should apply to .defineCatalogHandler too, and if the streams from
// kissanime don't expire in some way, we could use the same logic for .defineStreamHandler too

This would drastically reduce the number of requests your add-on is doing.

BoredLama commented 5 years ago

In-memory caching is not a must though, this could also be done in other ways, as alternatives (that i have worked with before) i would recommend sqlite or redis.

rleroi commented 5 years ago

I thought about saving everything to a database as well, and possibly connecting episodes to a IMDB ID so it can show up in the Cinemeta addon. Do you have more information about that? All I can find is this: https://github.com/Stremio/stremio-addons/blob/master/docs/tutorial/using-cinemeta.md and it seems to be outdated. One way would be to search stripped anime names on a movie database API. Maybe you know of another way?

Anyway, I might apply this in-memory optimization for now.

BoredLama commented 5 years ago

Yes I do, but I wouldn't recommend it..

To get IMDB ID of items, you can use: https://github.com/Ivshti/name-to-imdb

var nameToImdb = require("name-to-imdb");

// you can set type property too in the request, it can be 'movie' or 'series'

nameToImdb({ name: "south park" }, (err, res, inf) => { 
    console.log(res); // prints "tt0121955"
})

After you have the IMDB ID of your item, to get the Meta Object for it from Cinemeta, you can do:

var needle = require('needle') // or axios, or whatever
var type = 'movie' // type can be 'movie' or 'series'

needle.get('https://v3-cinemeta.strem.io/meta/' + type + '/' + imdbId + '.json', (err, resp, body) => {

    if (body && body.meta) {
        // success, body.meta is the meta object
    } else {
        // failed, cinemeta doesn't have your item
    }
})

This works perfectly for movies and series, but i STRONGLY advise against using this method for anime.. The reason why I advise against it is because IMDB is the worst possible DB for anime (and Cinemeta is based on IMDB IDs), it has very few of them.

Because it has few anime, and because there are also movies and series based on anime (but not anime), it will result in a lot of false positive results. Meaning that your anime will be matched with movies of the same name (in case there are any), or even totally different movies or series (if imdb doesn't have the anime at all).

The best DB for anime (and the only note-worthy one actually) is MyAnimeList, but they don't have any API (that I know of, at least), the only way it could be used is with html scraping (which is the worst way to use anything).

If you want my opinion, stick to kissanime API for this, it guarantees correct matches. And let's hope Stremio gets a Anime only DB based on a good anime source.

rleroi commented 5 years ago

Thanks for your thoughts, the problem is that some users cant find the streams. I've put in the add-on description now that they should go to Discover -> Series -> Anime to find them.

By the way, are you sure stream urls don't expire? Some streams are hosted on Facebook (One Piece (sub) EP2): https://video.xx.fbcdn.net/v/t42.9040-2/10000000_1304567209686507_4531495104950566912_n.mp4?_nc_cat=105&efg=eyJybHIiOjM2NywicmxhIjo0MDk2LCJ2ZW5jb2RlX3RhZyI6InN2ZV9zZCJ9&rl=367&vabr=204&_nc_ht=video.xx&oh=23f7e6572c47b25773de11341ca1054b&oe=5C4275CC When removing the URL params, the server responds with Bad URL timestamp, suggesting there is a time based hash in there. I'll try requesting the same stream again tomorrow to see if it has updated.

BoredLama commented 5 years ago

ah, their using the facebook cdn, yeah, those totally expire i never said they don't, i said "and if the streams from kissanime don't expire in some way", as there was a great chance they would expire.

rleroi commented 5 years ago

ah sorry, I misread your sentence ;)

BoredLama commented 5 years ago

the problem is that some users cant find the streams

you mean they don't see the catalogs? that's strange.. they should at least be visible in the Board page.. maybe some users just expected a "Anime" category in Discover? next to Series and Movies?

the only way to know the issues of users is to milk them of all the info they can give 😅

BoredLama commented 5 years ago

btw, u said u checked this link: https://github.com/Stremio/stremio-addons/blob/master/docs/tutorial/using-cinemeta.md

you should know that the new repo for stremio addons is: https://github.com/Stremio/stremio-addon-sdk/

the stremio-addons repo is completely outdated, it was for the previous addons sdk that got deprecated

also, if you would make issues about the trouble you're having with your add-on (instead of making comments in the code) with a little more data, maybe even a test case for the issues, i'd be more then glad to try to help with all the issues of this repo, until it's completely stable.

I've been hoping for an anime add-on in Stremio for a long time.

BoredLama commented 5 years ago

And you can check out my add-ons too if you want, their local add-ons though.. (you need to run them locally for them to work)

But their pretty awesome:

Adds Jackett support in Stremio. It brings torrent results from any of your favorite torrent sites, you can choose what torrent sites you want to enable in Jackett and the results show in Stremio. Jackett supports hundreds of torrent sites: public, private and even national torrent sites

This uses Google Search to find episode and movie streams from Open Directories (these are basically people that made their folders available online as a site, and forgot to set user / password, then google indexed them, lol).

You won't find the newest stuff on Open Directories, but it's amazing for old episodes and movies that are otherwise hard to find.