leighmacdonald / mika

Modern bittorrent tracker designed for private site use with multiple storage backends
MIT License
117 stars 17 forks source link

Replace http.query with faster & cleaner solution #6

Open leighmacdonald opened 4 years ago

leighmacdonald commented 4 years ago

The http.query functions "borrowed" from chihaya could be replaced with something cleaner for our needs. Since we are not necessarily trying to 100% conform to all of the bittorrent specs we can take some shortcuts that otherwise would not make sense for a public tracker.

It needs to be faster of course or there is no reason to change. Merges need to be supported by benchmarks for these critical hot code paths.

jzelinskie commented 4 years ago

Hey there! 👋 I'm a developer from the chihaya project.

I'm not a lawyer and our developers have no problem with sharing this code, but I'm fairly certain that you need to retain the copyright notice either in this file or at the root of the project. We'd love to collaborate further than sharing code, though!

Chihaya actually started as a private tracker and at one point had an administrative REST API, but when it had no usage whatsoever, we decided to drop support instead of developing something in a vacuum. In the current design, we tried to carve out extension points where said behavior could be implemented, with the hopes that we could eventually integrate this behavior extensions. It'd be phenomenally helpful if you could give us some feedback on what we could do to chihaya to make it something that could act as a foundation for something like this project.

The http.query functions "borrowed" from chihaya could be replaced with something cleaner for our needs. Since we are not necessarily trying to 100% conform to all of the bittorrent specs we can take some shortcuts that otherwise would not make sense for a public tracker.

We've tried to do our best conforming to the BitTorrent specs, but, realistically, the client behavior trumps the standards. This code is already taking shortcuts for the sake of performance compared to something like url.ParseQuery. Considering there are actually very few client libraries to support, I suspect any further optimizations could be shared across projects.

leighmacdonald commented 4 years ago

Hey Jimmy, glad to see you here! Ive loosely followed the progress of your project over the years. It's great stuff.

Ive added a copy of the license in the code header for the meantime.

That's very interesting to hear about the lack of interest or use in your API, I did not realize you guys used to have one actually implemented.

When this project started 6-7 years ago it was designed specifically for use in a couple greenfield tracker frontend projects which i had been asked to help with at the time. Because of my involvement on both sides of the stack we had somewhat tight coupling to the use of the API. I can certainly see how that's a bit of a luxury so it makes sense that you dropped support if you saw no use. There was a couple other projects that implemented mika, and i believe they mostly just use redis directly to manage things.

This project came back to life recently after seeing more interest in its use and is currently undergoing a big rewrite to fix a lot of design flaws in the old semi-private codebase. It was my first go project at the time so it was pretty rough and not very idiomatic at all.

I certainly have no problem sharing any improvements i can come up with to the parser, if any, back to you guys. Looking at your implementation now, it now it looks like you've already changed it from what i am currently using. The version i have was from at least 5 years ago when i last was working on this.

Ill certainly look into seeing how chihaya could better be leveraged as a base for something like this in the future, once i get further along. Its certainly changed a lot in those years so i am not totally up to speed on what it provides now.

jzelinskie commented 4 years ago

If you'd like to chat, our core developers always idle on #chihaya on freenode (which is also linked to #chihaya on matrix.org). We might be able to help answer questions or you could bounce ideas off of us while you work on the project. 😀