feature request: add support for tor HashedControlPassword authentication

[lesinigo]

TL;DR

Having support for HashedControlPassword authentication to the tor control port would be useful. The MESSAGING:onion section could have a new tor_control_password setting and use that if present or fall back to Cookie based auth if the setting is missing or empty.

rationale

The onion message channels currently support only Cookie based authentication, precluding (or, technically, making it much more difficult) to use a tor service running on another host / vm / container.

There could be a number of reasons to not want to have tor running alongside JoinMarket, in my case I want to have a number of services, like a bitcoin full node, a joinmarket yield generator, etc... running in isolated containers without any access to internet because I want all my traffic to go over tor.

This leads to these two issues:

1. when there are other services that need to use tor, running in other systems / containers, it would be a waste for every one of them to have its own tor, they could all use a single tor accessed over the network
2. if the services are supposed to only talk to the external world over tor, the network where services run would not have internet access (eg. no router / gateway) to enforce that policy. A dual-headed system / container could run tor, listening for connection over one network and accessing internet over a second network

For an actual example, my setup is using a docker-compose.yml along these lines:

---
networks:
  wan:
    # this has internet access
  tor:
    # no internet access
    internal: true

services:
  tor:
    networks:
      - wan
      - tor
  bitcoin:
    # the full node
    networks:
      - tor
  joinmarket:
    # could be a tumbler (uses tor but doesn't need auth to control port)
    # or yield generator (does need to authenticate with tor control port)
    networks:
      - tor

[PulpCattel]

to use a tor service running on another host / vm / container.

This is very much something I'd like to see, and I think it was mentioned in #1182 or its predecessor.

This leads to these two issues:

I would add a third, which is having Tor and other applications running on the same machine/vm/container is fundamentally less secure. Any compromised application can mess-up with Tor, and vice versa.

or yield generator (does need to authenticate with tor control port)

To be clear, as you say this feature is only useful for makers. Takers can already connect to Tor running on other machines/vms/containers as long as the Tor proxy (not control port) is exposed to them.

For makers, we could go a step further and also allow "static" onion service support. That is, the onion service is generated directly on the Tor machine/vm/container and no Tor control port is required at all.

I.e, add something like this to torrc config file.

HiddenServiceDir /var/lib/tor/hidden_service/
HiddenServicePort 80 10.152.152.11:80
HiddenServiceVersion 3

And then point JM to the generated onion service.

This is nice because the Tor control port is AFAIK potentially quite dangerous, as a compromised "non-Tor" machine/vm/container could learn our IP through it.

dangerous features [of the Tor control port]. The answer to the Tor control command GETINFO address will be the real external IP of the Tor client. Other dangerous commands include SETCONF, LOADCONF, GETCONF, GETINFO ns/id/

From https://whonix.org/wiki/Dev/onion-grater#Introduction

The downside would be that the onion service would persist across JM restarts, but this is not necessarily a problem in the first place (makers are already uniquely distinguished by their fidelity bond), it's trivial to regenerate a new onion service from Tor if needed, and maker bot are generally supposed to keep running for long period of time anyway.

[AdamISZ]

Yes, thanks to both for the discussion. I can also point you at #1315 as a reference.

It's not hard to see why some people find this desirable.

(I would first note that since the start of this repo, we've had the possibility to run all the messaging part (i.e. network communication, without any access to bitcoin keys or knowledge about bitcoin) by putting the jmdaemon code on a separate machine; the old setup written up by qubenix did this (see here though note it will be out of date).)

Second, about HASHEDPASSWORD, as far as I can see, it should be possible to use this with the existing codebase. To start, see:

https://github.com/meejah/txtorcon/blob/0c416cc8fe18b913cd0c7422935885a1bfecf4c0/txtorcon/torcontrolprotocol.py#L882

Next, here are two of the places we actually connect to Tor: jmbase.jmbase.twisted_utils.JMHiddenService.start_tor or jmdaemon.jmdaemon.onionmc.OnionPeer.connect

Next, note that txtorcon.connect has a keyword argument password_function which you can add as a callback so that when the control negotiation requests the password it will be taken from there (that's as it seems it should work; I haven't tested it).

About IRC: I now notice that currently the [MESSAGING:onion] section of the config does indeed have a tor_control_host and tor_control_port configuration (and it is also (duplicated!) in the BIP78 Payjoin section), but I fear that it does not exist for the old IRC style message channel configurations. If I've read that right it might mean that we need to upgrade/change that for the old IRC stuff as long as it exists (I don't see us dropping it quickly, yet).

If someone wants to PR this then, the simplest thing is probably just add the above keyword argument to our calls to txtorcon.connect and then test it with the onionmessaging (or if they feel like doing some more work, also patch up IRC to make sure we can connect to a remote Tor there, also; should be in jmdaemon.jmdaemon.irc.py).

[lesinigo]

@AdamISZ afaik the "IRC style message configuration" only does outgoing connections through tor (to the IRC servers) and does not need to publish any port for incoming connections, so it does not need the tor control port at all.

Yes, the MESSAGING:onion already has entries for host/port of Tor control and they are indeed working correctly already, it's just that the control connection only supports cookie authentication and not password auth.

@PulpCattel avoiding the control port at all and only using a static configuration would be much better, great idea! That said, I still think that if JM wants to keep supporting usage of the Tor control protocol and also to allow a non-local tor running on another host, it should also allow using HashedControlPassword as per this feature request.

[AdamISZ]

afaik the "IRC style message configuration" only does outgoing connections through tor (to the IRC servers) and does not need to publish any port for incoming connections, so it does not need the tor control port at all.

Oops my bad, good point! There is no need for inbound there, so no issue.

On that, apologies to @PulpCattel as I didn't actually get round to reading his detailed response, part of which referred to that distinction (more on that below).

Yes, the MESSAGING:onion already has entries for host/port of Tor control and they are indeed working correctly already, it's just that the control connection only supports cookie authentication and not password auth.

Yes of course. The rest of what I wrote was explaining how you could add it (not 'you' as in you, I mean anyone who feels inclined).

@PulpCattel

For makers, we could go a step further and also allow "static" onion service support. That is, the onion service is generated directly on the Tor machine/vm/container and no Tor control port is required at all.

Yeah that's a really good point. I had to write some awkward monkeypatched code to address a valid concern another user had about this control port access (maybe you already know, but just for reference: #843 ).

However I'm not exactly sure what the patch looks like for your alternative proposal; it seems a shame to not use ephemeral onion names (which, as it stands, is just one less permanent 'name' in the flow that makes things a little more ambiguous; your point about FB is right ofc and has been discussed at length, but I see that more as a flaw that might eventually get solved).

[PulpCattel]

@lesinigo

it should also allow using HashedControlPassword as per this feature request.

Yeah, absolutely. I said "also" too in my message, sorry if it wasn't clear. My ideal would be all 3 options (current, your proposal, static) available and user picks his favorite.

@AdamISZ

However I'm not exactly sure what the patch looks like for your alternative proposal

So, AFAIK it should be one of the easiest to implement (in theory at least). In this setup, conceptually JM doesn't even have to know Tor exists. Though in practice it might need to know the onion hostname, e.g., to advertise it.

The Tor docs explain it well.

https://community.torproject.org/onion-services/setup/

Step 1: Get a web server working As a first step, you should set up a web server locally

This would be JM running locally as per usual.

Step 2: Configure your Tor Onion Service

The user performs the onion creation on his own, and what it does at the line:

HiddenServicePort 5222 127.0.0.1:8080 is to put JM server host/port that we have in config. Of course, the user should configure these values to match what he actually needs on his machine/vm/container.

# the host/port actually serving the hidden service
# (note the *virtual port*, that the client uses,
# is hardcoded to as per below 'directory node configuration'.
onion_serving_host = 127.0.0.1
onion_serving_port = 8080

this says that any traffic incoming to port 5222 of your Onion Service should be redirected to 127.0.0.1:8080 (which is where the JM server from step 1 is listening).

In our case, the virtual port is hardcoded to 5222 I think.

That's it. It seems we don't even need a Tor library for this. If needed, we could add a config that takes the onion hostname (i.e., xyz.onion) so the user can provide it.

In terms of code, this means as JM we can assume all the onion service part is already taken care of and from our perspective we are just running locally.

it seems a shame to not use ephemeral onion names

You are right, ephemeral is nice. One thing to note is that this setup does not prevent that, if the user wants he can recreate the onion service every time the maker bot restarts. This "static" setup probably wouldn't be the default, so we can assume a bit more knowledge from the users if they choose to use it.

In contrast to this downside, another possible future benefit is allowing takers to cache/store the maker onion services, and thanks to that totally or partially bypass the directories. Even just as a fallback mechanism, where if the directories are offline or are playing games with us, we can always do without them. With ephemeral onions this is more challenging.

[PulpCattel]

I tested out a little bit how to use a static, Tor generated onion service (pretty much what I explained here).

I managed to make it work with a tiny diff.

diff --git a/jmclient/jmclient/configure.py b/jmclient/jmclient/configure.py
index d6b9af7..2d02500 100644
--- a/jmclient/jmclient/configure.py
+++ b/jmclient/jmclient/configure.py
@@ -559,7 +559,7 @@ def get_mchannels(mode="TAKER"):
                     ("socks5_host", str), ("socks5_port", int),
                     ("tor_control_host", str), ("tor_control_port", int),
                     ("onion_serving_host", str), ("onion_serving_port", int),
-                    ("hidden_service_dir", str)]
+                    ("hidden_service_dir", str), ("onion_service_hostname", str)]

     def get_irc_section(s):
         server_data = {}

diff --git a/jmdaemon/jmdaemon/onionmc.py b/jmdaemon/jmdaemon/onionmc.py
index 77ce1b2..af897bc 100644
--- a/jmdaemon/jmdaemon/onionmc.py
+++ b/jmdaemon/jmdaemon/onionmc.py
@@ -666,12 +666,21 @@ class OnionMessageChannel(MessageChannel):
         # we can direct messages via the protocol factory, which
         # will index protocol connections by peer location:
         self.proto_factory = OnionLineProtocolFactory(self)
         if self.onion_serving:
             if testing_mode:
                 # we serve over TCP:
                 self.testing_serverconn = reactor.listenTCP(self.onion_serving_port,
                                     self.proto_factory, interface="localhost")
                 self.onion_hostname = "127.0.0.1"
+            elif configdata['onion_service_hostname'] != None:^M
+                self.onion_hostname = configdata['onion_service_hostname']
+
+                self.serverconn = reactor.listenTCP(self.onion_serving_port,
+                                    self.proto_factory, interface=self.onion_serving_host)

             else:
                 self.hs = JMHiddenService(self.proto_factory,
                                           self.info_callback,

This small patch adds a new config (empty by default would be reasonable):

onion_service_hostname = foo.onion

If this config is set, then we skip the entire txtorcon stuff. Ideally, that dependency could then be optional. On signet, with this setup, I was able to run a maker from a Debian machine with no Tor and no access to Control Port, and I was able to partecipate in a CoinJoin with a taker of mine using sendpayment.py -P.

This is a quick and dirty POC, I didn't care much about the details, rather about showing the idea. @AdamISZ does it make some sense? I'm not familiar with Twisted, almost surely I'm doing something stupid (~in particular, I suspect it's not actually listening anywhere, but that's the only thing we have to do~ EDIT: Ok, now I can see from the taker log I'm successfully sending the handshake to my static onion service. Probably still not the right way to do it, but at least this should confirm the maker onion service is reachable).