kodifitzwell / repo

11 stars 0 forks source link

[FEATURE REQUEST] add Debrid Media Manager #16

Open hcgiub001 opened 2 days ago

hcgiub001 commented 2 days ago

hey, hope your well. So I've been using DMM for a while for Asian content (kdrama) and Anime have you heard of it? I've started searching inside of it for almost everything even if I have sources , the quality of the magnets it some times have given me much higher files. But for kdrama it's unmatched

Here is an example, person asking for https://www.reddit.com/r/Addons4Kodi/comments/1ff6pyw/good_addon_for_chinese_animes_like_battle_through/

Battle through the heavens, swallowed Star, Perfect world, Record of a mortal's journey to immortality, renegade immortal

DMM has all of them

It has an API https://github.com/debridmediamanager/debrid-media-manager

Someone mentioned that stremio actually uses Debrid Media Manager to pull sources so it seems their is a way? It also uses IMDB tt numbers as well as full title Thanks for your consideration

kodifitzwell commented 2 days ago

To make this process even easier, I've developed this free and open source website

it looks like a website in which login uses debrid authorization or with some magic it can be setup and run at ones own expense. is there an api?

DMM has all of them

not being sarcastic, how do you know? what kodi addon or software shows it?

battle through the heavens

which one are you referring to? the only one that has results on torrentio is tt12409194. what addon returns a result with the correct tmdb/imdb tags?

title year imdb tmdb trakt torrentio
Battle Through the Heavens 2017 tt12409194 -- -- link
Battle Through the Heavens 2022 -- -- slug --
Battle Through the Heaven 2023 tt30318518 241007 slug link
hcgiub001 commented 2 days ago

I can't find the exact comment I heard about the person who said that stremio addon uses DMM but I found this image https://www.reddit.com/r/Addons4Kodi/comments/1f33vc6/whats_the_difference_between_torrentio_on_stremio/

so did some searching and came across this https://elfhosted.com/app/comet/ the person who made DMM went on to make some other service which I have no interest but Scraping all public DMM content with Zilean so we are using public searches then https://github.com/iPromKnight/zilean

You're right I probably over stated that dmm has sources others don't, it's just I've been testing it over a couple of months. it's come up with whole series and movies that addons haven't been able to, since it's free and open source, people have used it to add magnets I'm guessing, then the site has a good database of the hashes.

The battle through heavens took me a while to figure out which one it was, I believe it's the first one you picked. that was one example. I'll curate a list for you if you interested.

If you still can't find a way or have the motivation , the site itself is pretty cool and would recommend to use it for personal stuff. I replied to your comment using DMM to find that magnet. It also really shines with Asian stuff, but if it's out on netflix, it's a good chance DMM will have it.

Since I've used it, it's never been down and one time I wanted to impress someone about it, it's down for some reason lol, if you give me some time to show you the results difference of what it pulls compared to torrentio, I'll make a list with lots of different content.

hcgiub001 commented 2 days ago

I think I got excited when going through it's github https://github.com/debridmediamanager/debrid-media-manager

External Search API If you don't want to build your own library, edit the config EXTERNAL_SEARCH_API_HOSTNAME in your .env.local and set it to https://corsproxy.org/?https://debridmediamanager.com

Docker Swarm cp .env .env.local docker swarm init docker stack deploy -c docker-compose.yml The website will be accessible at http://localhost:3000

xavengedx commented 2 days ago

Hello @hcgiub001 and @kodifitzwell

Sorry to chime in, but I'm very interested in DMM also as a search engine on Kodi! I've been using the site for a while, but I search the site and add it to my RD cloud, then when I do a search on Fen Light/POV, it appears at the top of my search as RDCLOUD! I have Comet installed on my Stremio and it searches for the DMM, below is an image as an example.

Screenshot_20240924-164001 (1).png

hcgiub001 commented 2 days ago

this is comet github https://github.com/g0ldyy/comet

image so it seems whatever comet is doing to pull DMM as a source , it's using Zilean https://github.com/iPromKnight/zilean

kodifitzwell commented 1 day ago

I have this mostly done; the number of results from one source is impressive. I'm 99% sure this is the way to do it, but there's 1% of me that wants to try another way.

the only real issue I have come across is what I believe is rate limiting, due mostly to supporting tv show packs. does DMM/Stremio show any indication that a result is a pack or support scraping packs? but then again it could be in testing I am just going down the list scraping too fast. real use case probably wouldn't be scraping anywhere near as I am in such a short period of time.

how does DMM or Stremio handle searching, imdb id or title?

web

body

xavengedx commented 1 day ago

Only @hcgiub001 can answer you! I don't understand ! But now I'm really looking forward to this update! It will be incredible to have the search in DMM, thank you for the excellent work.

hcgiub001 commented 1 day ago

Impressive work getting it done so fast.

I'll walk you through how DMM site is used, I'm not sure if you have access to the full site

this is homepage

image

when I go into search image

I can type as though I'm searching for a title or I can put in an imdb image

using matrix imdb image

now I'll show game of thrones image

we can add the whole pack, it also allows you to see inside the pack image

I'm not sure if this is what you are asking, but DMM does allow us to with one click install a single magnet that has mutiple files inside.

maybe I understand what you are asking, when we are searching , it says

filter results, supports regex

the top file in game of thrones is 274GB , I wanted to see if we can search for a single episode within the file (or within the pack), I found an imdb number for an episode I got a 404 this page could not be found.

I looked inside the first file (the biggest one) and copied one of the titles image

and nothing came up image

which means you can't search for individual files within a pack, and therefore it's not working the same as torrentio. you can search for individual seasons though if someone searched for game of thrones and wanted to add the first file, how would they do it? what I normally do is just add the whole pack, I also have it so it searches my online storage then it does allow me to select the individual files within.

The files that Kodi addons can't see (because of naming mismatch), I learnt of a FEN feature (not Fen Lite)

it's called link movie / tv show image

I'm hoping to get that feature back on addons. It basically always looks into that file for episode matching. I use FEN when I need that, that POV or Fen Lite can't reach

so answering your question, I'm not sure if we can search for individual files inside a pack and therefore add the pack once we are inside it, you can only add the whole pack, then search inside that pack later

you'll have to think of a way, a new method of how we load sources, since if someone wanted to watch an individual episode that a pack has, I don't think it will be picked up from POV search. yes we have actual individual episodes also and they will still see them, but I don't think we will see the the whole pack

maybe a suggestion, say I search for game of thrones and come across the poster for game of thrones, let us bring up context menu and search for the series game of thrones, so we can see packs from DMM and say I pick season 3 let us rick click on a season and "search DMM for season" so I can search for season 3 pack once we have clicked inside a pick we are adding it.

another way is you have a completely new search button just for DMM and you have methods of search that suit how DMM shows the results. I'm not sure about this, I don't know enough of programming and what you can see.

Really excited you planning to even add something, yes DMM is quite impressive, I've been meaning to make a video on how people can add magnets using it, but had hopes addons could add it directly, didn't think you'd do it so fast! I say take your time, and work out how you want to proceed. This is actually going to be beneficial for DMM which would be beneficial for KODI community to, since accessing files resets the cache deletion timer. thanks for considering this and working on it! let me know if I can help you in some way

hcgiub001 commented 1 day ago

one thing to note on their github, someone raised an issue https://github.com/debridmediamanager/debrid-media-manager/issues/131 Fix: Use better Anime metadata engine. #131

image

so they are still using metadata for other sites to categorise their own content? just thought it might be of help on showing you this for some reason

hcgiub001 commented 1 day ago

I did the search for MADAME WEB using POV image

compared to what you showed using DMM image

it not only pulled in almost twice the results, it did it faster and had higher quality than what torrentio pulled in. Now you can see why I wanted this so badly and why I always check DMM if it has a higher source than Torrnetio, and it's a good chance it has at least as good if not better. update is going to be great!

kodifitzwell commented 3 hours ago

an update has been pushed with the comet scraper. when it works, you get a load of results. movies are mostly not a problem, however shows have me scratching my head a little. disabling the pack search helps a lot, but for now I have left it enabled.

the way the addons/scrapers are coded now, for what it deems a probable ended season/show, it calls the api/website once each for the specific episode and season and/or show (so up to 3 calls). with comet, at any given time you might get a fast response, a looong response, no response, maybe only the episode, maybe only packs.

Dradis didn't seem to be affected as much, so I tried different methods of slowing the speed of the calls in POV because I think calling it up to 3x too fast caused the issue. but ultimately disabling pack scrapes seemed most effective.

the coded timeout is 15s, though the addon default scraper timeout is 10s so you might want to increase that.

my theory is some kind of rate limiting. in testing, I only had comet enabled. maybe this won't be as much of an issue in real use. this is using the comet "free community instance," a paid subscription is offered and I wonder if that is any less symptomatic.

I'm guessing this why the public instance of knightcrawler never really got much attention, which supposedly has torrentio and dmm hashes.

from my notes:

comet itself returns a "playable" url for each result item, that's why it requires the debrid api token. I believe this is dangerous because it is tied to the ip used to generate that url. there is an actual warning near the bottom of this page. playing that url automatically adds to the users cloud and resolves. so if one were to play that url and at the same time a housemate played something from a torrentio result, the debrid would probably see it as different ips. I'm not sure how stremio deals with this.

I disregard the url provided from comet.

I could mostly do the scrape without including the api token, but occasionally would get an error message if not included. my theory is there is an index that returns most results, when it couldn't find it in the index it had to manually check the debrid and without a token, received an error.

final thoughts:

torrentio/bitsearch is faster (though less results) for me and provides just enough results. so they will remain my only enabled default scrapers. comet is probably good to have for the "resrape with all" options.

I just like to watch, so the status quo is good enough for me. but someone with more time and motivation could do alot and create an addon (or modify existing with relatively less work) that would mostly mimic the dmm website. this was a quick and dirty solution.

for now, will leave this open for comments, issues...

kodifitzwell commented 3 hours ago

cmovies

cshows

hcgiub001 commented 1 hour ago

paid subscription is offered and I wonder if that is any less symptomatic.

Seems probable

comet itself returns a "playable" url

what's the difference from a url and other links that say torrentio returns?

I just like to watch, so the status quo is good enough for me. but someone with more time and motivation could do alot and create an addon (or modify existing with relatively less work) that would mostly mimic the dmm website. this was a quick and dirty solution.

you've done more than I wanted. and I thank you for your work. Even just for movies it's not a bad resource to have as a backup. I will be doing some testing and checking mostly for foreign content, also I'm curious if the results differ much from the results on DMM. Either way this is great news. thanks

Just to be clear, I'm okay using POV just with Comet scraper and at the same time another person on the same IP is using POV normally using Torrentio?

I'll report back on testing

kodifitzwell commented 38 minutes ago

what's the difference from a url and other links that say torrentio returns?

torrentio does not return a link, just some formatted information. we take some of that info and ask a debrid service if they have that cached. if they do, then we ask for a resolved link to play.

I'm okay using POV just with Comet scraper and at the same time another person on the same IP is using POV normally using Torrentio?

like torrentio, comet returns similar formatted information. we take some of that info and ask a debrid service if they have that cached. if they do, then we ask for a resolved link to play. comet also provides a playable link (that uses your debrid info) within that information, but I wanted to be transparent that the addon does nothing with that.

with my initial testing, I wasn't even sending my debrid token and was getting responses with urls. for that and performance reasons I believe comet uses your token only when you use the playable link. also, see the graphic just above the warning in that page. where they say play movie (using the returned link), we skip that and get the link from debrid ourselves.

so it should be, but as always use at your own risk.

I will be doing some testing and checking mostly for foreign content, also I'm curious if the results differ much from the results on DMM.

I will be interested in this too... comet probably won't be 1:1 but should be close.