Integrating i2p more closely with the Web Interface

[eyedeekay]

I noticed that the current i2p web interface is down. It seems that we currently expose the web interface to i2p by configuring i2ptunnel to forward the local port to the eepsite. This is fine, but kind of inconvenient and harder to back up, reconfigure and migrate. An essentially identical process is possible to do automatically by passing the tunnel configuration to the SAM API bridge, see my application sam-forwarder, which is in most ways a re-implementation of i2ptunnel as a SAM library. The advantage is that our configuration will be entirely separated from the actual i2p router, which will require us to set up less, it will allow Temporal to manage how it generates stores the i2p keys and consequent i2p address, making for automatic i2p configuration and much easier backups. If we lose our entire i2p router configuration in some bizarre accident, or want to migrate to a new i2p router, then we don't need to do any different configuration. All we need to have is a working SAM API and Temporal will configure itself.

[bonedaddy]

. If we lose our entire i2p router configuration in some bizarre accident,

Well it looks like this finally happened. While trying to migrate our ethereum node to a new disk in ESXi, I accidentally wiped out our I2P router, and guess I had forgotten to backup the configurations.

In the process of rebuilding our I2P router, and I think it'll be worth revisiting this. We recently refactored our playground, and it is much more I2P friendly now.

[eyedeekay]

I've got a whole bunch of small components(And one really big one, an embeddable router in the i2pd-related branch of checki2cp) that stand to drastically reduce the amount we need to think about keeping our I2P tunnels going. As a first step I'll collate them and figure out which ones matter for Temporal.

[bonedaddy]

Awesome that would be fantastic. Do you know if its possible to create a "high availability hidden service tunnel"? While setting up the hidden service tunnel I was able to point to one backend, however we have 2 different backends each with 3 different IP Addresses, is this possible to use via the i2prouter GUI? Or would this require the I2P stuff you've made?

Also is a simple rsync --recursive /var/lib/i2p remoteuser@remotedestination/backup good enough to backup the i2p router and its related configurations (including tunnels)?

[eyedeekay]

In an I2P context this might actually be a job for "Multihoming" which is where you use the same private keys for service tunnels on different routers with the same data behind them(How complicated this is to actually pull off is related to how much effort you have to put into keeping the data behind them identical, otherwise you're just spoofing your own site). Basically it makes answering on a tunnel a sort of race, where the router that answers first actually serves the request. Sort of a poor-man's load balancer(The same thing is possible with Tor AFAIK, where the phrase poor-man's load balancer was first used to describe this phenomenon on their network). idk.i2p and git.idk.i2p, for instance, are both multihomed on regular Java routers. It does require two routers to work, which in ideal situations would be on different hardware and in a truly perfect situation, they would be on different internet connections as well, but this is pretty close to what you're trying to do. More deliberate forms of load balancing can be combined with multihoming too, we could multihome a connection to like a reverse proxy or something that connects to one or the other of the backends. There could be some advantages to doing it that way, especially if it got bigger. @zlatinb was working on something called GarlicFarm which may also be worth exploring.

rsync --recursive /var/lib/i2p remoteuser@remotedestination/backup should be adequate to backup all the required information to completely migrate the router and all it's tunnels. Relating to Multihoming, this will also copy the i2ptunnel keys and i2ptunnel.config.d files required to set up a tunnel with identical keys on a different router. Personally I think the easiest way is to examine the i2ptunnel.config.d file to determine which key to use when cloning the tunnel, then copy that i2ptunnel.config.d file and that key to the new router. Alternatively, you could set up the tunnel via the GUI, stop it, then copy the new key over the one I2P generated when you set up the tunnel. In any case, make sure the permissions match, but it's all pretty straightforward.

[eyedeekay]

There are diminishing returns on multihoming, however, probably don't want to do it more than 4-5 times per i2ptunnel.

[bonedaddy]

Interesting, multihoming will be something i check out. I think we should be okay because all data is replicated, and the same database is used across all services unless there's an outage and we failover. I'm not sure how the multi-routers will work since everything will be on the same VM, behind the same internet connection but I'll do some research an experimentation on the i2p router I have at home. In the mean time i think a good solution is to stick an nginx reverse proxy on the i2p router, and have that forward to the appropriate backend services.

GarlicFarm seems pretty interesting, funny that you bring it up lol I was looking at the proposal thread on the i2p forum for that earlier today

Alternatively, you could set up the tunnel via the GUI, stop it, then copy the new key over the one I2P generated when you set up the tunnel. In any case, make sure the permissions match, but it's all pretty straightforward.

Sounds good, I'll backup with rsync and verify via the gui that all appropriate keys are as they should be.

[zlatinb]

Hi,

yes Garlic Farm is the name of the effort to bring "real" multihoming to I2P. But it's not yet finished and the code is not even on GitHub or git.idk.i2p, only in monotone. We will be migrating away from Monotone within the next two months so Garlic Farm should become visible, unless @eyedeekay wants to migrate it sooner.

The idea basically is that there is a "master" I2P destination and pair of keys, but then each node participating in the cluster has it's own destination and key pair. The master LeaseSet contains the LeaseSets of the cluster participants. In theory it can contain other master leasesets to create a tree structure, but at the moment Garlic Farm intends to support only one level of multi-homing.

Let me know if you have any further questions

[bonedaddy]

GarlicFarm sounds really cool, and exactly what I would be interested in for our services. Are you talking about the i2p monotone service? If so I can take a look at setting up the client, no need to rush migrating until you all have it planned.

[zlatinb]

Yes, it's in branch "i2p.farm" in the monotone service. Monotone doesn't work at all on recent Mac OS versions or on any Ubuntu newer than Bionic, don't know about Windows. If that's not going to cause too much hassle for you go for it; otherwise I can get zzz (the main dev) to export a tarball for you to poke around.

[bonedaddy]

Awesome, should be good virtualbox ftw! I will take a stab at this later tonight or tomorrow. If worst comes to worst I'll post here for a tarball but would rather not give zzz the extra work if its not required. Looking forward to checking it out

[zzzi2p]

Garlic Farm hasn't gotten any attention since last summer. We got bogged down in the complexity and we also didn't have great visibility into the requirements a large company would have for a highly-available, redundant service. It's not abandoned but there are now several higher-priority things in front of it.

The great @eyedeekay could pull you a tarball out of mtn for you in a minute, you don't need me to do it. But it's only the low-level machinery in the router that would be required. Proposal 150 is a higher-level view, but even that doesn't have any pictures of the overall system view. Additionally, Garlic Farm requires some features of Proposal 123 (LS2) that are implemented but not widely tested.

idk is correct above that simple multihoming works fine, to an upper limit of maybe 4-5 servers.

tl;dr we'd be happy to work with you on Garlic Farm but it's not going to be here soon.

[bonedaddy]

Quite understandable, all good things take time :smile:

The great @eyedeekay could pull you a tarball out of mtn for you in a minute, you don't need me to do it.

No worries, I'll sign up with monotone later tonight/tomorrow and download the tarballs, no need to pull people off their work just to get me a tarball, quite alright but much appreciated.

it's only the low-level machinery in the router that would be required. Proposal 150 is a higher-level view, but even that doesn't have any pictures of the overall system view. Additionally, Garlic Farm requires some features of Proposal 123 (LS2) that are implemented but not widely tested

I'll do some more thorough reading about those proposals

idk is correct above that simple multihoming works fine, to an upper limit of maybe 4-5 servers.

That should be good for now

tl;dr we'd be happy to work with you on Garlic Farm but it's not going to be here soon.

What language is Garlic Farm written in? If it's in Go I'd be happy to contribute some code where I can, if not I can be a guinea :pig2: and help with testing