Connect to Tor and use Ephemeral Onion Services

[felipedau]

As requested by @adrelanos in #4, unMessage could use the Tor ControlProtocol to create Ephemeral Onion Services to facilitate running it on Whonix. These changes use other (new) features of txtorcon in order to achieve that.

unMessage now only allow users to choose from two options: launching its own Tor process or connecting to an existing one. The former is the default behavior and the latter is what should be used for Whonix. After unMessage is able to control a Tor process (either by launching or connecting) it creates an Ephemeral Onion Service for the peer. Once it is created, the private key is added to the peer information so that it can be persisted and the same Onion Service can be recreated.

One of the good things of using Ephemeral Onion Services is that a confirmation is sent after its creation. That makes this process blocking so that now unMessage only starts when the Onion Service is really available. This solves the issue when users launch unMessage at the same time and try to connect to each other, while their Onion Services are not available. Unfortunately it takes longer to launch unMessage, but at least it is guaranteed that the peer is really online.

One issue that still has to be fixed is when users initialize connections to each other simultaneously. Now that right after unMessage is launched the Onion Service is ready to receive connections, when both users have presence enabled, two connections are made at the same time.

These changes will also require newer versions of Tor (0.2.9) and txtorcon (0.19) so we need to make sure users can easily install them. As soon as a new txtorcon release is made it will be automatically installed, but updating Tor would still be manual.

Also, as the database structure changed to persist the Onion Service's private key, these changes are incompatible with the previous file. It might be easier to just delete the peer's dir within ~/.config/unMessage and restablish conversations.

Let me know what you think!

Note

This branch uses unreleased code from txtorcon's master branch and will only be merged to develop until txtorcon 0.19 is released.

[adrelanos]

https://github.com/felipedau/unmessage/tree/feature/ephemeral-onion-services

---

I appreciate that you are signing your commits!

Where can I find your gpg key D83961D42653880D?

I have 0xC5A49047572A0D47 but that is not the one your commits are signed with.

git log --show-signature

commit 04924518eb95d1596c68c5b1bccb8b8379ee28a3
gpg: Signature made Thu 23 Feb 2017 01:41:34 AM UTC
gpg:                using RSA key D83961D42653880D
gpg: Can't check signature: No public key
Author: Felipe Dau <dau@riseup.net>
Date:   Thu Feb 23 01:37:38 2017 +0000

    Check if there is an Onion Service before removing it

    It is possible to not have an Onion Service when on local mode, but
    also when unMessage fails to add one and does not crash.

[felipedau]

Where can I find your gpg key D83961D42653880D?

I have 0xC5A49047572A0D47 but that is not the one your commits are signed with.

Sorry @adrelanos, I had not published that! The one you have is my personal key.

Here are the keys: my commits are signed with my software signing key and the tags are signed with anemone's software signing key.

Thanks!

[felipedau]

Just an update:

The changes from 791448a remove the dependency on txsocksx and uses new code from meejah/txtorcon#216, so who is testing this branch should update it as well.

Thanks!

[felipedau]

811bf58 allows using the GUI on Whonix. I do not know how to present these options to the user yet. I think that maybe adding too many "distractions" to the graphical interface will confuse users. So at the moment I am leaving these options hidden, and when we get it to work fully on Whonix we make some kind of alias/shortcut/config to use them automatically without exposing to the user. @adrelanos, what do you think?

The GUI is able to be launched now, just as you used to launch the CLI:

unmessage-gui -n n --local-server-ip $(qubesdb-read /qubes-ip) --connect-to-tor -c 9151 -t 9050

Thanks!

[adrelanos]

--connect-to-tor

Why do we need --connect-to-tor? Do you otherwise start Tor? Then I'd suggest... tor-prompt logic is "start Tor if not already running". And with latest anon-ws-disable-stacked-tor, tor-prompt detects Tor already being running. (Even if Tor is not actually running. A stub process is which does just sleep.) Perhaps "do like tor-prompt" or "do like onionshare"?

-c 9151 -t 9050

You could just assume default localhost and default localhost ports. anon-ws-disable-stacked-tor would be doing the proper forwarding.

--local-server-ip $(qubesdb-read /qubes-ip)

Will be sorted out when we sort out https://github.com/AnemoneLabs/unmessage/issues/2.

[felipedau]

Why do we need --connect-to-tor? Do you otherwise start Tor? Then I'd suggest... tor-prompt logic is "start Tor if not already running". And with latest anon-ws-disable-stacked-tor, tor-prompt detects Tor already being running. (Even if Tor is not actually running. A stub process is which does just sleep.) Perhaps "do like tor-prompt" or "do like onionshare"?

That's mostly because we did not put enough work on that to make a clever solution such as the ones you mentioned. As by default users would not be able to control the main process and we did not want to make them configure authentication, we chose to launch unMessage's own process by default. In Whonix' case, it is very simple to just connect to the process, but not the case for most systems. So yeah, --connect-to-tor isFalse by default, but I really want to make unMessage smarter on these situations.

I'm not not sure if this makes sense: but I felt that launching a new process makes the actions kind of sandboxed and specially for an app at the stage that unMessage is, avoiding harm to the main tor process is important (I wouldn't want it to break other peoples' tors). Am I completely wrong about this?

You could just assume default localhost and default localhost ports. anon-ws-disable-stacked-tor would be doing the proper forwarding.

Right! As I mentioned above, currently there is not a logic dealing with this and the default ports we chose to use are 9055 and 9054 which will be available in most systems so that we can launch a new process using them. That's the reason when connecting to an existing process you have to provide other ones.

--local-server-ip $(qubesdb-read /qubes-ip)

Will be sorted out when we sort out #2.

By the way, are you planning to set a "deadline" which is when people will have the opportunity to give feedback until we start to implement it? I've been checking that list every other day but it's been a while since someone commented.

Thanks for the suggestions, I am going to open a ticket for that as well!

[adrelanos]

Felipe Dau:

Why do we need --connect-to-tor? Do you otherwise start Tor? Then I'd suggest... tor-prompt logic is "start Tor if not already running". And with latest anon-ws-disable-stacked-tor, tor-prompt detects Tor already being running. (Even if Tor is not actually running. A stub process is which does just sleep.) Perhaps "do like tor-prompt" or "do like onionshare"?

That's mostly because we did not put enough work on that to make a clever solution such as the ones you mentioned. As by default users would not be able to control the main process and we did not want to make them configure authentication, we chose to launch unMessage's own process by default. In Whonix' case, it is very simple to just connect to the process, but not the case for most systems. So yeah, --connect-to-tor isFalse by default, but I really want to make unMessage smarter on these situations.

I'm not not sure if this makes sense: but I felt that launching a new process makes the actions kind of sandboxed and specially for an app at the stage that unMessage is, avoiding harm to the main tor process is important (I wouldn't want it to brake other peoples' tors). Am I completely wrong about this?

I absolutely understand the desire of not having the user to configure Tor control cookie authentication. That's a usability stopping point (many users would not get past that point). Perhaps look if onionshare found a solution with good usability for that? (I use onionshare only in Whonix. Cannot comment on onionshare usability for lets say Debian systems with system Tor [the tor package from packages.debian.org].)

Starting your own Tor process imo is a troublesome approach. By starting your own Tor process, user torrc configuration changes are not adhered. Someone who configured bridges for example bridges to hide connecting to the public Tor network would suddenly and unexpected connect to the public Tor network.

Also subverts Tor's entry guard mechanism.

Not exactly the same topic, but somewhat related... We've been wondering at Whonix if we should use a Single Tor-Gateway with Multiple Workstations or Multiple Tor-Gateways mapped 1:1 to Workstation VMs. Not an easy question. There might be a Tor blog research wanted post sometime soon on that topic. Reference:

https://www.whonix.org/wiki/Dev/Multiple_Whonix-Workstations

To get Tor stream isolation, which is highly recommend, set random socks passwords unmessage_ per user account. (Dunno if there is multi account support implemented or upcoming.) Then you could benefit from Tor's default IsolateSOCKSAuth.

Just my thoughts on that. I'd recommend to consider double checking with someone from The Tor Project or so.

You could just assume default localhost and default localhost ports. anon-ws-disable-stacked-tor would be doing the proper forwarding.

Right! As I mentioned above, currently there is not a logic dealing with this and the default ports we chose to use are 9055 and 9054 which will be available in most systems so that we can launch a new process using them. That's the reason when connecting to an existing process you have to provide other ones.

Would be simple to add redirection of localhost ports 9055 and 9054 to the gateway in Whonix. Would be just two more lines of config in anon-ws-disable-stacked-tor

Might be a good idea to make anon-ws-disable-stacked-tor smart enough to start in such cases a fake instance (just a shell wrapper for compatibility). Does unmessage just run 'tor --some-args'? That should even work already. args would be ignored but the processes (which does just sleep) would be started.

--local-server-ip $(qubesdb-read /qubes-ip)

Will be sorted out when we sort out #2.

By the way, are you planning to set a "deadline" which is when people will have the opportunity to give feedback until we start to implement it? I've been checking that list every other day but it's been a while since someone commented.

No deadline. Just need to find time to continue working on it. That is incorporating the feedback and posting it again.

[felipedau]

I absolutely understand the desire of not having the user to configure Tor control cookie authentication. That's a usability stopping point (many users would not get past that point). Perhaps look if onionshare found a solution with good usability for that? (I use onionshare only in Whonix. Cannot comment on onionshare usability for lets say Debian systems with system Tor [the tor package from packages.debian.org].)

onionshare's wiki has guides for Mac, Windows and Linux, but all require changing torrc. There is one very simple solution (for Linux) which adds the user to debian-tor but I do not think that is a good idea.

Starting your own Tor process imo is a troublesome approach. By starting your own Tor process, user torrc configuration changes are not adhered. Someone who configured bridges for example bridges to hide connecting to the public Tor network would suddenly and unexpected connect to the public Tor network. Also subverts Tor's entry guard mechanism.

Wow that's a great point! Although I think we could read torrc and use the same options for this new process, it wouldn't be that simple to make it use the same entry guards, right?

Not exactly the same topic, but somewhat related... We've been wondering at Whonix if we should use a Single Tor-Gateway with Multiple Workstations or Multiple Tor-Gateways mapped 1:1 to Workstation VMs. Not an easy question. There might be a Tor blog research wanted post sometime soon on that topic. Reference: https://www.whonix.org/wiki/Dev/Multiple_Whonix-Workstations

Thanks for the link, that's very interesting and certainly not easy. I prefer the idea of using multiple processes on the same Gateway mostly because, as mentioned, multiple Gateways would make the usability worse and the host machine resources requirement would get much higher which can make a barrier for users.

Also, if people need to change Tor's settings, there is a chance of making a mistake and using the wrong vm.

To get Tor stream isolation, which is highly recommend, set random socks passwords unmessage_ per user account. (Dunno if there is multi account support implemented or upcoming.) Then you could benefit from Tor's default IsolateSOCKSAuth.

Although that "feature" has not been tested enough, it is possible to use multiple accounts. That is a great idea. Thanks!

Just my thoughts on that. I'd recommend to consider double checking with someone from The Tor Project or so.

Thanks, I will!

Would be simple to add redirection of localhost ports 9055 and 9054 to the gateway in Whonix. Would be just two more lines of config in anon-ws-disable-stacked-tor

Might be a good idea to make anon-ws-disable-stacked-tor smart enough to start in such cases a fake instance (just a shell wrapper for compatibility). Does unmessage just run 'tor --some-args'? That should even work already. args would be ignored but the processes (which does just sleep) would be started.

That would be nice, although I think that once #31 is implemented, we can use different ports in different situations and that should not be needed. I wouldn't want to have anon-ws-disable-stacked-tor changed because of that, and which would only fix it for Whonix, but also because unMessage should automatically work on multiple systems. I think we can fix that "just from unMessage's side". What do you think?

No deadline. Just need to find time to continue working on it. That is incorporating the feedback and posting it again.

Got it. Let me know if there is anything I can do to help.

[adrelanos]

Felipe Dau:

I absolutely understand the desire of not having the user to configure Tor control cookie authentication. That's a usability stopping point (many users would not get past that point). Perhaps look if onionshare found a solution with good usability for that? (I use onionshare only in Whonix. Cannot comment on onionshare usability for lets say Debian systems with system Tor [the tor package from packages.debian.org].)

onionshare's wiki has guides for Mac, Windows and Linux, but all require changing torrc.

There is one very simple solution (for Linux) which adds the user to debian-tor but I do not think that is a good idea.

Personally I think on a single user system it's not a big issue. When the user account get compromised, all application data is compromised. Having access to Tor doesn't make things much worse then.

Problem is rather on Debian policy level. Adding user to debian-tor group is something a system administrator is free to do. I am pretty sure however that Debian packages may not add group debian-tor to user 'user'.

(We do that in Whonix, but as a derivative we can bend the policy and just do it to improve usability.)

Starting your own Tor process imo is a troublesome approach. By starting your own Tor process, user torrc configuration changes are not adhered. Someone who configured bridges for example bridges to hide connecting to the public Tor network would suddenly and unexpected connect to the public Tor network. Also subverts Tor's entry guard mechanism.

Wow that's a great point! Although I think we could read torrc and use the same options for this new process,

And (on Debian) process /usr/share/tor/tor-service-defaults-torrc as well and don't miss when /etc/torrc.d [1] gets implemented.

[1] https://trac.torproject.org/projects/tor/ticket/1922

it wouldn't be that simple to make it use the same entry guards, right?

Right.

Would be simple to add redirection of localhost ports 9055 and 9054 to the gateway in Whonix. Would be just two more lines of config in anon-ws-disable-stacked-tor

Might be a good idea to make anon-ws-disable-stacked-tor smart enough to start in such cases a fake instance (just a shell wrapper for compatibility). Does unmessage just run 'tor --some-args'? That should even work already. args would be ignored but the processes (which does just sleep) would be started.

That would be nice, although I think that once #31 is implemented, we can use different ports in different situations and that should not be needed. I wouldn't want to have anon-ws-disable-stacked-tor changed because of that, and which would only fix it for Whonix, but also because unMessage should automatically work on multiple systems. I think we can fix that "just from unMessage's side". What do you think?

That would be awesome!

No deadline. Just need to find time to continue working on it. That is incorporating the feedback and posting it again.

Got it. Let me know if there is anything I can do to help.

If you like, you could incorporate the feedback from the debian-devel mailing list and update the draft accordingly.

[felipedau]

Personally I think on a single user system it's not a big issue. When the user account get compromised, all application data is compromised. Having access to Tor doesn't make things much worse then.

Hmm, from that new perspective, I agree with you. Also, both solutions require higher privileges and adding the user to debian-tor or not won't make a difference once the (sudoer) user is compromised and privilege escalation will be a matter of time.

Problem is rather on Debian policy level. Adding user to debian-tor group is something a system administrator is free to do. I am pretty sure however that Debian packages may not add group debian-tor to user 'user'.

As apparently all the solutions (to use the main process) require higher privileges, we (un)fortunately depend on the admin. So maybe in those cases we can only launch a new tor?

And (on Debian) process /usr/share/tor/tor-service-defaults-torrc as well and don't miss when /etc/torrc.d [1] gets implemented.

[1] https://trac.torproject.org/projects/tor/ticket/1922

Thanks! I was not aware of that.

If you like, you could incorporate the feedback from the debian-devel mailing list and update the draft accordingly.

Alright, just posted an update to the original ticket.

Thanks @adrelanos.

[felipedau]

As apparently all the solutions (to use the main process) require higher privileges, we (un)fortunately depend on the admin. So maybe in those cases we can only launch a new tor?

Well, that's not true because I forgot that the easiest solution presented by onionshare is to use TBB (which makes me wonder if it uses the main process or launches its own, respecting torrc and using the same entry guards).

I do think (after reading your posts) that launching unMessage's own process should be the last attempt.

[adrelanos]

I don't have a good answer. It's somewhat boils down to security vs usability.

Are you at the Tor meeting or perhaps any future Tor meeting? Perhaps that would be a great place to ask?

https://trac.torproject.org/projects/tor/wiki/org/meetings/2017Amsterdam

[felipedau]

It's somewhat boils down to security vs usability.

Indeed!

Are you at the Tor meeting or perhaps any future Tor meeting? Perhaps that would be a great place to ask?

Well, I wish I was involved to the Tor community (I'm also considering applying to this year's GSoC), but I don't see myself attending Tor meetings too soon. If you do have the opportunity to ask, that would probably be the best place to do so! We could always ask on irc or mailing lists though. Should we compose a question? Here is what I have in mind:

What are the best practices for apps that need to use a Tor process? Should they attempt to acquire one that is currently running (and which one? system, TBB, etc) or launch their own?

Or should we compose an actual post with a few paragraphs containing more info/context and why we got to this point?

Thanks!

[adrelanos]

tor-dev mailing list is the best place to ask, I think.

Composing a question is a good idea.

Additionally, when I am at the Tor meeting, I'll happily ask a few people. For that I just need bullet points so I don't forget any major points.

[felipedau]

tor-dev mailing list is the best place to ask, I think. Composing a question is a good idea. Additionally, when I am at the Tor meeting, I'll happily ask a few people. For that I just need bullet points so I don't forget any major points.

Awesome! Let me know how that goes and then we can compose a post for tor-dev.

Thanks!