search

boramalper / magnetico

Autonomous (self-hosted) BitTorrent DHT search engine suite.
http://labs.boramalper.org/magnetico/
GNU Affero General Public License v3.0
3.06k stars 343 forks source link

Peer / seeder counts? #18

Open dessalines opened 7 years ago

dessalines commented 7 years ago

I noticed other dht indexers have this, is this available here? When there are 20 torrents of the same thing, I need to know which one is most seeded.

boramalper commented 7 years ago

We need to implement BEP 33 for that. I will definitely look into it, thanks for your feedback!

scriptzteam commented 7 years ago

Much, much needed :D

Maverik44 commented 7 years ago

Good idea

mweinelt commented 6 years ago

DHT scrapes are too expensive, tracker scrapes should be preferred for bulk scraping.

kescherCode commented 5 years ago

DHT scrapes are too expensive, tracker scrapes should be preferred for bulk scraping.

You do know that magnetico's goal is to function without trackers?

jangrewe commented 5 years ago

Using the internet is also too slow, sending hard drives per express courier should be preferred for fast data collection.

kescherCode commented 5 years ago

Hard drives are too slow, would rather send solid state drives.

anacrolix commented 4 years ago

In all seriousness, BEP33 could allow this data to be collected with existing queries, and responses stored in the DB.

anacrolix commented 4 years ago

@boramalper it looks like there's columns in the database, and constants in the API to support seeder and leecher counts and querying, but I don't see any code actually using it anywhere. It appears that querying on these causes a panic, and that none of the DHT exploration code fills in the database values. Could you confirm that's the case? Are there any PRs/existing attempts to do this, or was it scoped out in the boilerplate, but nobody got around to it? Cheers

anacrolix commented 4 years ago

I've implemented BEP33 upstream, and started trying to integrate it with magneticod. As far as I can tell, magneticod inserts complete torrents into the database only when it has the metadata, it has no provision for updating or merging peer/seed counts over time. It looks like some fields where intended for this like updated_on, but were never wired up. I assume it would require some redesign or an additional service within magneticod to continuously maintain an up-to-date value for n_seeders and n_leechers.

anacrolix commented 4 years ago

It's not pretty, but it works as a proof of concept: https://github.com/boramalper/magnetico/compare/master...anacrolix:popularity?expand=1.

kodxana commented 3 years ago

Any updates on seed/peer counter :)

dessalines commented 3 years ago

I built a library that does this a few years ago that I'm using in torrents-csv: https://github.com/dessalines/torrent-tracker-health