bitmagnet-io / bitmagnet

A self-hosted BitTorrent indexer, DHT crawler, content classifier and torrent search engine with web UI, GraphQL API and Servarr stack integration.
https://bitmagnet.io/
MIT License
2.01k stars 76 forks source link

Option to Return Real-Debrid Cached Items/Hashes #35

Open dreulavelle opened 7 months ago

dreulavelle commented 7 months ago

Have you checked the roadmap on the website, and existing issues, before opening a duplicate issue? Yes

Is your feature request related to a problem? Please describe. I'm apart of several communities that typically only have 2 options when it comes to searching for cached torrent/magnet hashes on a given Debrid service. Options like Orionoid or Torrentio are allowing us to create Plex/Jellyfin/Emby libraries quickly with responses of cached hashes on a given debrid service.

Currently I've written a Torrentio indexer for Prowlarr that's been the closest thing to integrating the *Arr apps with Real-Debrid. We could take this a large step further if we had an application that searches the DHT as a whole and returns results based on "cached status".

Describe the solution you'd like I think something like a parameter at the end of the bitmagnet url, something like &cached=real-debrid would filter the results by what's been cached on real-debrid (or any of the debrid services). A real-debrid, or other debrid service, api-key would have to be given as well.

Describe alternatives you've considered As mentioned before, my way of circumventing this was writing a custom indexer for Prowlarr to achieve this. However Torrentio has limitations, and I imagine I may see those same limitations in Orionoid too (I'll be working on this one soon).

The solution I feel would be an "on-demand" cache checking application that can search many indexers, and return only cached results.

Additional context Hopefully this falls within scope of your project. Here are a couple resources to look at that may be helpful.

mgdigital commented 7 months ago

Sounds interesting. Real Debrid is something on the periphery of my awareness. Am i right in saying this request could be rephrased as:

Can I have the ability to add any arbitrary tag to a torrent (either manually or via the API) and then be able to search on those tags (or have a filtered Torznab output based on them)?

You could then have some other process running setting the appropriate real-debrid-cached tags.

My intuition is that a smallish subset of users would be interested in the Real Debrid element (or even know what it is), while ability to add arbitrary tags would be generically useful and allow others to implement stuff like this themselves in separate services.

If I can be convinced there's enough demand then the RD API itself might be more tightly integrated, but would (probably) still make use of arbitrary tagging rather than having its own space in the app's domain model.

dreulavelle commented 7 months ago

I'm sure you've heard of Kodi, and even Stremio.. and there's a couple other communities well over 1000+ users that all use real-debrid. It's very tough to beat the RD service as a whole because everything is cached in their database.

A user doesn't have to download a file locally to use it, if it's cached, it's simply added to their account instantaneously and is available within seconds to be accessed. You can access it by HTTP or by WebDAV. Additionally, Rclone can also be used to mount these locally as well. Plex/Emby/Jellyfin/Infuse users can also access real-debrid this way and see your full library on RD.

Kodi users use plugins that have the capabilities of RD being integrated to check hashes on demand with other non-cached indexer.

Stremio users typically use the Torrentio addon as well. Which can return cached options as well.

The population and demand is definitely there for Real-Debrid. I can however understand if this falls outside of the scope of what your trying to accomplish.

Currently the RD community just scrapes other indexers, and then checks every hash and filters. Returning only cached hashes back to the user. But the problem is, is that many apps already do this with only utilizing a couple indexer at most. Or they've integrated it into their own database (like Torrentio for instance, or Orionoid). Being able to search the DHT though and only returning cached results based on a keyword or imdbid/tmdbid or whatnot, could be a game changer.

Hmm.. let me know your thoughts! Thank you

mgdigital commented 7 months ago

You've made it sound compelling and I'll do some more research on RD - however there is much work to be done on the app's current core features before the scope could be widened to include this.

A feature that is likely to be implemented in the nearer term is:

Can I have the ability to add any arbitrary tag to a torrent (either manually or via the API) and then be able to search on those tags (or have a filtered Torznab output based on them)?

So combined with this:

You could then have some other process running setting the appropriate real-debrid-cached tags.

Would that get you at least part of the way there for the time being?

If others are interested in seeing a tighter RD integration happen they should 👍 @dreulavelle 's original post and it will be prioritised accordingly.

dreulavelle commented 7 months ago

That could work really well. What else would be a tag?

Things like torrent/content information like...

This could work well for the api, could sort by multiple tags.. title + year + cached status or something of the sort..

Ultimately I'm trying to mitigate the use of regex as much as possible so that apps know EXACTLY what a file is.. whether its a movie, or a tv show .. or an anime movie/show.. or a kids movie. This makes integrating things into Plex and similarr apps much easier.

edit: damn typo's... lol

mgdigital commented 7 months ago
  • Title
  • Year
  • Quality movie, or a tv show .. or a movie.. or a kids movie.

These don't need to be tags - they're already part of the core domain model.

Title, year content type (movie etc) and genre can already be searched, filtered and aggregated via the API (year filtering is not in the UI yet but is available in the API and is also part of each item's searchable string so will also work with the UI search box). Have a look at the GraphQL playground by going to http://your-bitmagnet-instance/graphql in your browser to see what's currently available.

"Quality" can mean a few things - we already have source (BluRay etc).and resolution (1080p/4160p etc) in the API and UI. If you mean bitrate then an imperfect proxy would be filesize divided by runtime (both already in the API but not filterable, I think the *arr apps already do this too).

What else would be a tag?

Tags could be used for many things such as manual content curation, fill in gaps in functionality such as your RD feature request, and generally allow users to enrich their index with info that isn't already part of the domain model. That's why it would be a generically useful feature.

dreulavelle commented 7 months ago

"Quality" can mean a few things - we already have source (BluRay etc).and resolution (1080p/4160p etc)

I was "summing" up these 2 options lol. Source/Resolution as "Quality". But this is great! Eventually when the federated "feature" get's implemented, we can all piggyback off each others hash databases.

Haven't played with GraphQL before, so that'll be exciting! :D