Open sgmihai opened 1 month ago
I agree it's hard to follow when used as a guide to help you set it up from start to finish.
Look these are the conclusions I made after running various layouts with tens of users and several gigabits/s traffic.
"You are gonna need 1 or more hetzner 2 core vm's for internal and 1 upto 4 hetzner 1 core vm's for external (RRDNS) (SSL offloading) and 1 or more accounts from https://whatsmyuse.net/ for every internal backend backbone."
first of all, what would be the benefit of this layout you imagined people are going to use, one box for connection to nntp servers, and external machines that you connect to, to do the actual downloading ?
This decision depends if you will be running a public or private news service.
If your public IP is known you run the risk of account closure due to abuse.
To have a different IP doing NNTP you run an external + internal layout.
"Clients are load balanced using ip hash, client accounts use the same internal always so your other internal backbone accounts are protected. :-)"
don't get it.
This decision is important when running a public news service.
Imagine a specially crafted NZB file which only has articles available on 1 specific backbone. Nobody else has this NZB.
This NZB is now flowing through your system and it triggers an alert. The exact backbone is used and nobody else has this NZB. Your account will be closed.
Many internals will protect your accounts as the NZB flows to the same internal.
"Destroy and re-create VM when reaching 20 TB to reset bandwidth and avoid paying the 1 EUR per TB surcharge."
Is there really no better way, sounds like a big hassle to recreate vm. Or more suitable provider for this ? 20TB is kind of low for non-casual users.
You can do whatever you want. This was just the cheapest way to get gigabytes/s having multiple internals and/or externals. Traffic is very expensive above 1 gbit ports you know.
In the meantime Hetzner has already made a few changes. Not sure if it's possible to do several gigabits/s anymore.
"I suggest 1 external and 1 internal to start out with to 'hide' the IP fetching the article. :-)" I guess this is the key to the whole concept. But I still don't get it. What IP to hide from whom ? and why.
Please see my reply to your first question.
Anyway, thanks a lot for the effort, I was hoping this exists. I don't plan to run my own "parasite" nntp service, although it would be fun :) Maybe when you enable local caching.
I have uncommitted code that supports local caching using S3. I tried it with R2 and it wasn't all that bad. 60 MB/s. I'm sure it can be written much smarter.
Thank you for the explanation. I only had in mind the scenario for sharing among friends mostly. Still, isn't it easy, once someone knows your service is of the parasite type, to signup, and simply get some random obscure nzbs and check what user downloads them on their backend at that moment ? Probably most usenet providers are in some kind of chat group somewhere, and they could collude to cross-check all of their logs for this activity. Then they can just disable the account.
I managed to get client user auth working, but for some reason I get this when trying to download "[backend] No backends found". https://i.imgur.com/T7aP8ig.png Any idea what's wrong with the config ? Also, I noticed in your screenshots the collation is utf8mb4_general_ci , while in the created tables it's ut8mb_unicode_ci. I am guessing you changed it in the meantime and this shouldn't matter, just leave it as it is ?
edit: Well nevermind, I figured it out, looked at the code and saw that node must be "2" for some reason.
edit: Well nevermind, I figured it out, looked at the code and saw that node must be "2" for some reason.
:-)
"You are gonna need 1 or more hetzner 2 core vm's for internal and 1 upto 4 hetzner 1 core vm's for external (RRDNS) (SSL offloading) and 1 or more accounts from https://whatsmyuse.net/ for every internal backend backbone."
first of all, what would be the benefit of this layout you imagined people are going to use, one box for connection to nntp servers, and external machines that you connect to, to do the actual downloading ?
"Clients are load balanced using ip hash, client accounts use the same internal always so your other internal backbone accounts are protected. :-)"
don't get it.
"Destroy and re-create VM when reaching 20 TB to reset bandwidth and avoid paying the 1 EUR per TB surcharge."
Is there really no better way, sounds like a big hassle to recreate vm. Or more suitable provider for this ? 20TB is kind of low for non-casual users.
"I suggest 1 external and 1 internal to start out with to 'hide' the IP fetching the article. :-)" I guess this is the key to the whole concept. But I still don't get it. What IP to hide from whom ? and why.
I will try to install and use it either way, but it doesn't feel right not understanding what the author meant :)
Maybe try to rewrite it in a more sequantial/logical way, less cowboy style.
Anyway, thanks a lot for the effort, I was hoping this exists. I don't plan to run my own "parasite" nntp service, although it would be fun :) Maybe when you enable local caching.