greatest-ape / aquatic

High-performance open BitTorrent tracker (UDP, HTTP, WebTorrent)
Apache License 2.0
478 stars 33 forks source link

Add Dockerfile for aquatic_ws #81

Closed zackees closed 2 years ago

zackees commented 2 years ago

Thanks for the great work here on acquatic.

Would you consider creating a docker file so that it's easier to test this software (and become a contributor) without having to go through the Rust setup?

greatest-ape commented 2 years ago

@zackees

I realise that being able to build and run aquatic_ws with Docker would be slightly easier (and require making less changes to your machine) than installing rustup and cmake and building it. However, creating a decent Dockerfile that covers things such as setting up a TLS certificate (a real one, or maybe a bogus one for debugging locally), mapping configuration keys and achieving decent build times is not trivial. I unfortunately don't think that the group of people who currently are not able to/interested in running aquatic, but would be if a Docker setup was available, is large enough for me to justify expending the time.

Developing Rust projects is generally a lot slower when using Docker since incremental compilation isn't supported, including when using cargo-chef and the like. Contributors will likely want to install rustc and cmake on their own machine (or a Linux VM) eventually.

However, if you want to take a stab at creating a Dockerfile, you could have a look at the CI pipeline. You could also build off something like the following:

# Run aquatic_ws in Docker (place in aquatic_ws/Dockerfile)
#
# $ docker build -t aquatic-ws -f aquatic_ws/Dockerfile .
# $ docker run -it --rm -p 3000:3000 --name aquatic-ws aquatic-ws

FROM rust:latest
WORKDIR /usr/src/aquatic
COPY . .
RUN . ./scripts/env-native-cpu-without-avx-512 && cargo install --path ./aquatic_ws
RUN ./scripts/gen-tls.sh
RUN echo "log_level = 'debug'\n[network]\naddress = '0.0.0.0:3000'\ntls_certificate_path = './tmp/tls/cert.crt'\ntls_private_key_path = './tmp/tls/key.pk8'" > ./ws.toml
EXPOSE 3000
CMD ["aquatic_ws", "-c", "./ws.toml"]
zackees commented 2 years ago

Sorry if this is newb question but why do you need to have TLS in the app itself?

When I deploy http microservices with DigitalOcean or render.com I always use port 80 and let the DO/Render.com service itself issue TLS/SSL certs. The app itself just communicates through 0.0.0.0:80 and has no idea what the actual domain it's resolving to, nor does it care. It's all being handled by the reverse proxy.

As far as barrier of entry: nothing is easier than a docker instance. I can point and click install apps from github. I'm looking to launch a movie using bittorrent to millions of people. I can't tell you which movie yet, but it's with an established director and this is the third movie in the series. I'm struggling because I'm an embedded C++ developer and transitioned recently to webdev. I'd consider myself a junior webdev and the amount of effort to get your project up and running is high.

Would you agree that having a dockerfile would move your tracker code from something a small sliver of developers are able to have the knowledge to install, to literally a no-code, point and click solution? If you had a docker install, why would anyone use any other tracker?

zackees commented 2 years ago

I'll donate $$ to the project if you can implement this feature with DigitalOcean as the reference host.

That's how important this feature is.

zackees commented 2 years ago

Here's a testing client fork of instant.io which adds specification for tracker server

https://webtorrentseeder.com

zackees commented 2 years ago

Fork: https://github.com/greatest-ape/aquatic

I've put about 8 hours into this with docker and the main blocker right now on both Digital Ocean and Render.com is the failure of the health checks.

image

Which from the documents shows that what's basically happening is that these services are sending a ping to the service at port 8080 (or 3000 in the case of this app) and the expected result is a 200 or 204 response code. It appears that the aquatic_ws doesn't have an http engine and so the request produces an error and the docker instance fails to deploy.

https://www.digitalocean.com/community/questions/why-am-i-getting-a-deploy-error-health-checks

zackees commented 2 years ago

I'm closing this bug since i was able to deploy a dockerized version of bittorrent-tracker. Although it's not nearly as fast as your rust based solution, it does get the job done and is accessible to a broader range of developers and even non developers.

So I'll go ahead and close this bug. Thanks for the quick response and taking the time to create a dockerfile!

https://github.com/zackees/docker-bittorrent-tracker

greatest-ape commented 2 years ago

Thanks for trying!

It would indeed be possible to make TLS optional, and if I do, I might also create a Dockerfile for a reasonable “very easy to deploy” default config (no TLS, assuming use of a reverse proxy, no access control). I see your point about lowering the cost of entry.

I would however then also have to consider if I want the extra complexity of responding to HTTP GET requests just for cases like the health checks that you mentioned, even if they are not part of WebSocket specification.

Then of course, people might still want to eg only allow announces for certain torrents (which I assume would indeed be important to you for DCMA reasons), meaning that the dockerfile would have to support that too.

Long story short is, we’ll see if I find it worthwhile.

zackees commented 2 years ago

I would however then also have to consider if I want the extra complexity of responding to HTTP GET requests just for cases like the health checks that you mentioned, even if they are not part of WebSocket specification.

I think this would be trivial to implement and very important. People want to see if the program works. You could just report "ok" or do something like print out stats.

Then of course, people might still want to eg only allow announces for certain torrents (which I assume would indeed be important to you for DCMA reasons), meaning that the dockerfile would have to support that too.

That would be a great feature idea. Would you need it though in v1?

greatest-ape commented 2 years ago

I would however then also have to consider if I want the extra complexity of responding to HTTP GET requests just for cases like the health checks that you mentioned, even if they are not part of WebSocket specification.

I think this would be trivial to implement and very important. People want to see if the program works. You could just report "ok" or do something like print out stats.

No, it’s not trivial, I’ve had a look at it.

Having a way to monitor the service definitely makes sense, but I’m thinking more along the lines of a Prometheus integration or the like.

Then of course, people might still want to eg only allow announces for certain torrents (which I assume would indeed be important to you for DCMA reasons), meaning that the dockerfile would have to support that too.

That would be a great feature idea. Would you need it though in v1?

Yes. The whole point is to make aquatic easier to use for people who do not know how to compile applications, run their own server etc. If they are hosting an open tracker on any major docker hoster such as render, their account will definitely be shut down if they track allegedly infringing hashes and can not meaningfully respond to DMCA notices (frankly, they are likely to have their accounts deactivated before they even have a chance to react to a notice, especially if running on a free tier).

I might be interested in implementing the functionality you’re looking for under contract. Feel free to send me an email if you want to discuss details :-)

zackees commented 2 years ago

If they are hosting an open tracker on any major docker hoster such as render, their account will definitely be shut down if they track allegedly infringing hashes and can not meaningfully respond to DMCA notices

For my uses, I am whitelisting the tracker to allow exactly one file. Later, I may add the ability to add more.