Closed nyxnor closed 2 years ago
https://tails.boum.org/news/test_4.20-rc1/ Over the years, we have identified many issues in how to connect to the Tor network from Tails, thanks to your feedback. With this release candidate, we are trying to solve the most important and pressing of these issues:
So, before Tor even starts, mask the service, ask to configure bridges or change other configs, but bridges is what matters
Thank you, nyxnor. You raise good points, but I have to look into it more in detail. The ping, for example, has not only the function to see if the domain is blocked - but the service could also be offline. Actually, that was the initial reason for this kind of check.
A very good suggestion is to use session.proxies['http']='socks5h://localhost:9050'
. However, that has to be checked in detail on side effects. We have to avoid a situation in which someone needs tor to fix the tor installation. Also, we have to keep in mind that TorBox also has to work with Captive Portals. This is why the local DNS resolution (not the ones from the clients) cannot be routed through Tor (see more details here).
In other words: very good suggestions, which we should look into, but this will not be done in a few weeks.
For whom it concerns Currently, I'm on travel, and for that reason, I don't have time to make basic changes. My focus is on a check & fix system, which detects a connection loss with the Tor network and which automatically initiates countermeasures (for example, the change of an entry guard or the restart of tor). If you are interested in the development, check out the "automatisation-test" branch.
Tor Browser starts with DisableNetwork 1
, so the first prompt is to configure bridges if needed or set other parameters such as proxy, security settings.
https://manpages.debian.org/buster/tor/torrc.5.en.html
DisableNetwork 0|1
When this option is set, we don’t listen for or accept any connections other than controller connections, and we close (and don’t reattempt) any outbound connections. Controllers sometimes use this option to avoid using the network until Tor is fully configured. Tor will make still certain network-related calls (like DNS lookups) as a part of its configuration process, even if DisableNetwork is set. (Default: 0)
But tor (daemon, service) does not come with this by default, so it automatically connects to the network.
Either place torrc in /etc/tor/torrc with DisableNetwork 1
before instaling the package (could already be done and I haven't noticed, as it is already in the path), or mask before installing with sudo systemctl mask tor@default.service
.
I guess the DisableNetwork 0|1
is a different topic, and I don't see the benefit it would have for TorBox.
Either place torrc in /etc/tor/torrc with
DisableNetwork 1
before instaling the package (could already be done and I haven't noticed, as it is already in the path), or mask before installing withsudo systemctl mask tor@default.service
.
What do you mean with "before installing the package"?
Just start tor service after configuring bridges, making it similar to Tails or Tor Browser way https://github.com/radio24/TorBox/issues/67#issuecomment-872917339 By disabling network, could achieve this, considering the topics expressed above.
Another good thing is to acquire bridges with moat https://gitlab.torproject.org/tpo/applications/tor-browser/-/blob/tor-browser-78.11.0esr-10.5-1/browser/modules/BridgeDB.jsm Well, if this could be translated to python would be good.
Another good thing is to acquire bridges with moat https://gitlab.torproject.org/tpo/applications/tor-browser/-/blob/tor-browser-78.11.0esr-10.5-1/browser/modules/BridgeDB.jsm Well, if this could be translated to python would be good.
This is already integrated. Entry 3 in the Countermeasure sub-menu gives the possibility to automatically add a OBFS4 bridge every 24h.
With moat? It is with HTTPS no? https://github.com/radio24/TorBox/blob/c27028f7a99c32e8a769a74e7760fc6860448764/bridges_get.py#L32 If domain is blocked, this won't work.
I don't know at the current time how to implement moat .jsm to python to be called via command line.
With moat? It is with HTTPS no?
https://github.com/radio24/TorBox/blob/c27028f7a99c32e8a769a74e7760fc6860448764/bridges_get.py#L32
If domain is blocked, this won't work. I don't know at the current time how to implement moat .jsm to python to be called via command line.
You are absolutely right... it is https. I will ask my Python guru, if he has any idea how to integrate that.
Smart torification
Old code: use clearnet
OCHECK=$(curl -m 6 -s $BRIDGE_DB_URL)
New code: first try to use tor, and if this is not working, use clearnet
#-m 6 must not be lower, otherwise it looks like there is no connection!
OCHECK=$(curl --socks5 192.168.42.1:9050 --socks5-hostname 192.168.42.1:9050 -m 6 -s $BRIDGE_DB_URL)
if [ $? == 0 ]; then OCHECK="0"; else OCHECK="1"; fi
if [ $OCHECK == 1 ]; then OCHECK=$(curl -m 6 -s $BRIDGE_DB_URL); fi
if [ $? == 0 ]; then OCHECK="0"; else OCHECK="1"; fi
if [ $OCHECK == 0 ]; then
Thank you for this: get bridges through tor in case of dns attack and Try to fetch bridges over tor
With the latest commit from Carlos Martinez, we turned it around: first, we try to fetch the bridge over tor and only if that fails then we use clearnet. In the script executing bridges_get.py, the nameservers were switched to public ones before. This means that if the script is using clearnet, the public nameservers will not be affected by cheap DNS related censorship.
We will implement the same for bridges_check.py
In the build script, you first acquire the keys from the torproject.org domain., then build from github, then add snowflake from the torproject.org domain. Acquiring the keys will fail if the domain is blocked, as well as the snowflake.
What about ordering the build script this way:
torsocks
torsocks git clone
.You can see this build example here 1, 2, 3, 4, 5 - All from here
If using the torproject.org plainnet domain, it will fail when the domain is blocked, see related issue here (it was using cloudflare 1.1.1.1 dns resolver and failing. What should be used, or at least an option to choose is to APT over Tor, my example here. That user is from Turkey, he has been using my build script and working for him, see PR for him here. Not merged mainstream on the other project due to the build script being programmed to change a lot.
Change the git repos of Snowflake to https://github.com/keroserene/snowflake.git, avoiding pinging torproject.org or acquire it with torsocks
.
See how you can change sources with this whiptail I made, sources here, the equivalent file of torbox.lib.
Can I do these changes?
In principle, I fully agree with you. Some of the points you raise are on my personal "smart torification" list, which I like to integrate for the upcoming version. However, there are also some differences and open questions between your approach and my current idea:
Although I may not accept all of your changes, it would be very interesting, helpful and time-saving for me if you could create such a pull request.
I also already studies your whiptail menu, although I wouldn't give the user too much possibility to interfere. My idea is that the script does the safest way, and if that fails, it tries an alternative method.
I cringe at my whiptail menus ( I tried to keep it super simple). But about user messing up the configuration, well, this is just giving them choices. Maybe when you build TorBox, you just set the security you want with it just like Tor Browser:
This can all be solved with one simple option, ./install/run_install.sh [normal|torified].
I didn't find the linked discussion comment that 1.1.1.1 would not work to circumvent a cheap censorship mechanism
It is inserted here. Although it was not discussed on that issue, I was chatting with that user privately to help him and he stated he was using.
I guess one significant difference could be that I use two variants of tor installation so far. One is fetched from the torproject.org source repo and the other from GitHub. With the GitHub approach, the user can select which version of tor he wishes to install. Although the goal is to torify the TorBox, I will not change the possibility of choosing between different versions.
Building from git is much better. I just mentioned to install from package manager cause from the Blitz project, it is to be faster, but for a Tor router, yes should focus on fetching from github.
What are the reason that you mask tor.
Yes, I did not explain here, but the reason to mask tor is to avoid making any connections to the Tor network before configuring bridges. If the user already has bridges, he can just uncomment that from the torrc and then let tor start with all traffic routing with bridges with pluggable transport, so no tries before a safe configuration. If the user does not have bridges, even though there are already 36 in the torrc sample, tor needs to be unmasked and acquired from the database with onion routing.
Although I may not accept all of your changes, it would be very interesting, helpful and time-saving for me if you could create such a pull request.
Let's discuss thoroughly what is needed so when there is enough information and agreement, I will start.
I'm thankful for all the inspiration you give to the project. Nevertheless, currently, it is a little bit hard for me to explain in detail what I want. Usually, I start with an idea and then it develops it further when I see the code in front of me. So, I began to torify the update process. For me, the best way is that first, the script tries to fetch all the necessary packages through tor. If that fails, it will switch to clearnet (it asks for permission to do so, first). The name server was changed to public ones before.
You can find parts of the new code here, here, here, and here
Summary The idea is not to route the local TorBox data through tor because that will break the captive portal capabilities and has other side effects if tor is not working properly on the TorBox anymore. However, I will torify - if possible - all data traffic for the maintenance, bridge fetching, bridge checking and so on. I guess this is the main difference between the Raspiblitz and TorBox.
Problem
I just finished looking at the install / build script for the Raspiblitz. Very well done and very impressive work. Also, I understand now the ordering of your build script. If you have time and motivation, I would be thankful, If you could work on run_install.sh for TorBox.
I can do that. update
script looking good!
Will add in the install script apt-transport-tor
, good to add here after apt-transport-https
.
I recommend having a review on that page, it is missing python packages too such as pillow and requests (requirements)
I recommend having a review on that page, it is missing python packages too such as pillow and requests (requirements)
I will update the page after the release of v.0.4.2. This is also the branch we working on.
Thoughts on implementation: Test if tor is working, if it is, try with torsocks and hidden service for all its benefits, if not working, either:
paranoia switch
.Thoughts on implementation: Test if tor is working, if it is, try with torsocks and hidden service for all its benefits, if not working, either:
- ping torproject.org plainnet.
- exit script warning it avoided to ping the domain? I don't know if you want this or not with
paranoia switch
.
I guess you are speaking about the installation script. If possible, it should be the same process as we have now in the update script. Test if we can reach the URL (.onion) through tor. If no, ask the user if he wants to connect through clearnet or not. If no -> exit.
At the end of the installation and on the image file, tor can still be masked. Therefore log in (SSH or web) is mandatory to start the initialisation function in the main menu script. There the user can be asked if he wants to activate bridges and to forbid clearnet access on tor related URLs. After he answered this question and the settings were set accordingly, tor can be unmasked.
Slight plan:
run_install.sh
# SYNTAX
# ./run_install.sh <--select-tor> <--circumvent>
#
# The <--select-tor> option allows the user to select a specific tor version.
# Without this option, the installation script installs the latest stable version.
#
# The <--circumvent> option will:
# * default to APT over Tor for the Tor Project packages
# * calls menu-bridges to configure bridges and bypass censorship
# Without this option, will ping https://deb.torproject.org for sources and no bridges by default.
if [ "${CIRCUmVENT}"== "--circumvent" ]; then
CIRCUMVENTb=1
SOURCES_TOR_UPDATE_CURL="--socks5 192.168.42.1:9050 --socks5-hostname 192.168.42.1:9050 http://apow7mjfryruh65chtdydfmqfpj5btws7nbocgtaovhvezgccyjazpqd.onion"
else
CIRCUMVENTb=0
SOURCES_TOR_UPDATE_CURL="https://deb.torproject.org"
fi
sudo sed -i "s/^CIRCUMVENTION=.*/CIRCUMVENTION=${CIRCUMVENTb}/" ${RUNFILE}`
sudo curl ${SOURCES_TOR_UPDATE_ONION_CURL}/torproject.org/A3C4F0F979CAA22CDBA8F512EE8CBC9E886DDD89.asc | sudo apt-key add -
menu
check_fresh_install
{
CIRCUMVENT=$(grep "^CIRCUMVENT=.*" ${RUNFILE})
if [ $CIRCUMVENT = "CIRCUMVENT=1" ]; then
clear
bash menu-bridges
online_check_update
set_sources_list
fi
}
In the update script, I turned it around: first, try over the tor socket, if that is not working, ask if using clearnet is ok. It may be better, we use the torsocket for all tor related URLs as default and not the other way around.
Implemented with TorBox version 0.4.2!
can't acquire new bridges or list their status just because the domain is blocked. Why not make the request over Tor? Possible responses for that:
questions to think of possible outcomes
1 - domain blocked
Well, reaching torproject.org over plainnet would the worst in this case, they already block that or even store the user query.
If Tor is not running not because of this, some config might be wrong.
session.proxies['http']='socks5h://localhost:9050'
basically, of course there is more involved. Example (1) (2)2 - tor connection blocked
Well, possibly the attacker also blocks the domain, so this method shouldn't be recommended, but instructed to add manually if got no connection to the database. Is it possible to now if it is blocked without trying? If user already knows, should a parameter be set to not trying tor connections without bridges?
mapping
domain blocked
tor connection blocked
also, there is already preset bridges in the torrc, so shouldn't have to worry about using the defaults if needed at first
can't connect for whatever reason