Tracker data scraping and display

[wiput1999]

I think it should have this following features

• Seed/Peer count
• Member System

Then improve performance I think Docker is a good choice to made it scalable.

Thanks, Wiput Pootong

[lukaspanek199]

Due to the lack of a tracker and no intentions in making one, I don't think seed/peer count will be possible. And for the member system, I don't know if nyaa had something like that

[egozi44]

Nyaa at least let you register and add subscribe system later on, include the ability to press on the torrent in order to see description, pic, and whatever the uploader wrote, and ofc commits below, will these come back too?

[ghost]

TPB doesn't have a tracker (only uses magnets) and still displays Seed/Peer count, how they do that i don't know. Doesn't display files, however; but i think it should be possible to implement, as per #25. IMO The Seed/Peer count feature is important, can't really be a proper indexer without that.

As for the member system, i think he's talking about how on old Nyaa you could register and upload torrents as an user, instead of anonymous upload, so people could track your releases. And the moderators, like i mentioned on #35, could mark some users as "trusted", so their releases always appear green.

[majestrate]

you can do a scrape from trackers that support it for each swarm. would have to be carefully limited to not annoy. could also do a dht spider approach, i.e. N dht announces per interval to guess activity.

[one-summers-day]

Can you guys paste all your feature requests here so me and my buds working on the frontend can try and sort things out? Thanks in advance :) https://docs.google.com/document/d/1MgAPqrVqggXji7EqdH8EcRuew83uZoEkGrSCDjhcXMI/edit?usp=sharing

[egozi44]

Will be nice if we could enter each torrent page like before and see description and other things similar to how nyaa was, that actually was one of the things that identified nyaa as different from the other torrent sites, he was more friendly and had a bit more options that gave people easier time uploading/downloading

[one-summers-day]

@egozi44 comment plz

[s-varezhkin]

Destroyed

[wiput1999]

@Jiro-Horikoshi I have added new topic about performance improvement I think If you do it scalable It will be fine for future :D

[ewhal]

We should have an option that the user can set for whatever tracker they want to scrape from. This'll allow people to add in trackers that they host themselves such as https://github.com/chihaya/chihaya Random tracker as an example. http://tracker.baka-sub.cf/scrape?info_hash=OLg%92MH%f8%5d%0f%a3%0f%ec%f4%9dh%ae%b7%c3%b0%04

[jhemmmm]

Try sharing your backup db. :)

[Roguyt]

@MaeJhem Instead of being so rude, maybe you could explain why they aren't doing well in your opinion and maybe they'll respond to your request about nyaa's db backup :)

[jhemmmm]

Oh my bad, it's not like I'm being rude. I'm just telling that the site isn't good YET. There's still a lot of features that are missing. try adding a bottom pagination. It's kinda hard to scroll up again to just go to the next page.

[ewhal]

It's still in alpha. https://kitsunemimi.pw/tmp/merged.sqlite3.zip / https://kitsunemimi.pw/tmp/merged.sql.zip http://66.11.126.121/sukebei.7z

[jhemmmm]

Great. I'll take a good look on this one. :)

[majestrate]

on the topic of chihaya perhaps consider adding a db driver backend to integrate with it

On May 6, 2017 3:18 AM, "Eliot Whalan" notifications@github.com wrote:

We should have an option that the user can set for whatever tracker they want to scrape from. This'll allow people to add in trackers that they host themselves such as https://github.com/chihaya/chihaya

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/ewhal/nyaa/issues/70#issuecomment-299621537, or mute the thread https://github.com/notifications/unsubscribe-auth/AAefxQmn4TanrAtTICI9EIaVK9ouUu6qks5r3B6sgaJpZM4NSWDJ .

[jhemmmm]

So, the database was ripped.

[egozi44]

@egozi44 comment plz

on?

[ewhal]

We have a tracker that the owner is happy for us to scrape from

[an-electric-sheep]

TPB doesn't have a tracker (only uses magnets) and still displays Seed/Peer count, how they do that i don't know.

There are several ways to scrape a tracker.

• UDP. can scrape 74 torrents at a time
• http full scrape. it's just /scrape without additional parameters, returns scrape information for all infohashes. rarely supported due to the large amount of traffic it can cause
• http multi scrape. you can pass multiple info_hash arguments to the scrape url and it returns scrapes for each. but there usually is an upper limit to how many you can get
• http single scrape. essentially useless for indexers
• non-standardized dump interfaces or static .tar.gz exports. you need to consult the homepage or contract the admins for individual trackers to get that one.

[wiput1999]

@ewhal I think your new nyaa is the best come back :D I will look forward and support as I can if in the future I can develop in Golang I will come to help you develop this project.