Closed nyxnor closed 3 years ago
Let's say I have a set of hidden services I want to add a set of clients, I can tick in a checklist the services I want to add clients, as well as well as add multiple clients at the same time, separated by a comma like this https://github.com/radio24/TorBox/blob/eb88bba0744b8ee0ec9bab6f232ce1faa4b16420/bridges_activate_old#L163 And if I loose trust a set of clients and want to remove them from the authorized list of multiple services, this should be done too.
Some scripts are required to be run with the debian-tor
user, as it reads the /var/lib/tor
to check if service or client exists. I don't think there is a way to avoid that.
also Please Add option to import and export hidden services and torrc file from home directory, because directly to /var/lib/tor cannot copy files from scp an example, and export too, need special rights., for my is complicated, because, need to edit torrc file.
@connected201 Thank you for this. import and export functions will be integrated to preserve onion addresses after flashing the sd card to update major versions of torbox, as well as importing previous services data from another computer.
Process should be to tar
the folders /var/lib/tor/onion_auth
and /var/lib/tor/services
, define remote host (ip, user, path) to scp (maybe store these variables).
A whiptail inputbox to register data and whiptai menu to click to export or import data.
Tasks
Current problem I need help to bypass. When selecting the option client auth from the menu, user can select multiple services. When that happens, I want to show the intersection of the authorization files with the same name that are present on different services dir.
ssh/authorized_clients/alice.auth
ssh/authorized_clients/bob.auth
test/authorized_clients/alice.auth
If selecting ssh and test service, I want to display only alice.
Keyworkds to search bash/shell
, intersection
, multiple
, files
, directories/folders
, find
, compare
.
https://en.wikipedia.org/wiki/Intersection_(set_theory)
I need to do with bash the intersection of multiple sets.
Okay, this seems to help https://stackoverflow.com/a/28161520
echo ${array1[@]} ${array2[@]} ${array3[@]} | tr ' ' '\n' | sort | uniq -c
count number of arrays and math with values that represent the same number. Concatenating arrays would also serve.
Taking the official documentation about setting up an Onion Service as a reference, are these files necessary for the client authorisation? If I understand it right, with it, the owner of the server can decide who can access it and who not. How is it with a public server - let's say torbox.ch on an onion address? Does such a server need these .auth files too?
Yes, those files are necessary for servers with client auth. 2019 guide and the clean community guide.
The nomenclatures I use for clarification: Hidden server - just known to user or to select group that was shared Public server - normally used with onion location, to be known and seen by anyone
Hidden servers are appropriate to set up Client Authorization to manage who can view the service and access it, without the client key, the service page does not even load, protecting the privacy and against DOS attacks.
torbox.ch is supposed to be seen and used by anyone, so it should not require a client key. I know it is hard to grasp without testing, but I did a lot of improvements to the script and will try to release them soon after enough testing.
More about client auth: The script I made automatically saves to the server authorized clients the pub key of each client, it echos the private key in the right format to be used in the torbrowser or to be saved to onion_auth/alice.auth_private, this is manual step by the client, you just need to pass them the key and directions, which will be done with infomation displayed with text/onion-auth-explanation-save-client-key
Take a look here https://github.com/nyxnor/onion-cli The checklist option for whiptail is too ugly, it is using that but with dialog checklist commented if needed to change.
Everything there has a reason, if it does not sound right, ask me. Manual inputs only happen 2 times, when creating a hidden service, cause it is necessary to type the service name and ports, the 2nd time it occurs is when adding clients, user need to give a name for them. Menu does almost everything that the tor.onion-service.sh does but with CLI it is limited, I think I reached 90% of what the menu does. What is missing is purging services after deleting their configuration or purging all clients from selected services (but this last can be done by selecting the desired services one by one and their clients). Purging the service data dir can be implemented with a warning: "Do you want to preserve the onion address or purge?", default to not delete. Will adapt this to whiptail:
echo "Client Private key for ${SERVICE}"
echo
echo "RAW:"
echo
echo "Address = "${TOR_ADDRESS}
echo "Key = "${PRIV_KEY}
echo "Conf = "${TORRC_CLIENT_KEY}
echo
echo
echo "EXPLAINED:"
echo
echo " BROWSER -> Typing the key in the GUI"
echo " * In the browser, enter the service address = "${TOR_ADDRESS}
echo " * A small window will be prompted, enter the key = "${PRIV_KEY}
echo
echo " BROWSER and DAEMON 2 -> Adding the key to torrc to be read automatically"
echo " * Add the line containing ClientOnionAuthDir to the torrc file accordingly to your setup (remove identation):"
echo " - Browser = [Tor_Browser_folder]/Browser/TorBrowser/Data/Tor/torrc"
echo " ClientOnionAuthDir TorBrowser/Data/Tor/onion_auth"
echo " - Daemon = /etc/tor/torrc"
echo " ClientOnionAuthDir /var/lib/tor/onion_auth/"
echo
echo " * Add the private key (note: same content for Browser and Daemon but different paths):"
echo " - Browser = [Tor_Browser_folder]/Browser/TorBrowser/Data/Tor/onion_auth/bob.auth_private"
echo " - Daemon = /var/lib/tor/onion_auth/bob.auth_private"
echo " "${TORRC_CLIENT_KEY}
echo
echo " * Restart the instance"
echo " - Browser = Close and open again the Tor Browser Bundle"
echo " - Daemon = Reload the daemon = $ sudo pkill -sighup tor"
echo
echo " * Go to the service address = "${TOR_ADDRESS}
The cleanest guide to understadn client auth: https://matt.traudt.xyz/posts/creating-private-v3-FgbdRTFr/ and the cleanest to host an onion service https://matt.traudt.xyz/posts/website-setup/
Updated repo https://github.com/nyxnor/onion-cli
It is very simple, the .md files are there cause I am planning on adding them to instructional part of the menu.
Onion-Location is just a guide.
Backup functionality such as export and import are not finished yet and need this to be complete (but not only this, constant bugs trying to solve). is now finished but I need to test thoroughly if it does not lead to bugs, this is the most crucial part, making sure to save the hs secret keys.
I recently started using Qubes-Whonix and it enhances the security of the system overall, a lot for onion services. Some parts of the scripts may not work on whonix (HiddenServicePort target as unix:path do not work because it is a VM), but I am using them as a security guide, even though debian is no whonix, we can harden debian too. Take a look here https://www.whonix.org/wiki/Onion_Services#Security_Recommendations
I made onion-cli
in a way it can be just git cloned and run, no installation of software (beside vanguards if requested). So it can be cloned to TorBox if needed. If you plan to make changes, please commit directly so I can better the software.
Demonstration video of some of the functionalities: https://twitter.com/nyxnor/status/1434033535678091270
Overall view of the TUI, it just calls the cli script. Also if you notice, on the TUI, it only appears more options that required a onion service if you have at least one configured. lib + tui + cli = 1500 lines of code, which is short to review.
very nice, it will be implement in next release of torbox?
Regarding the implementation, my idea is to do it in several steps:
How about the time table To-do for the next 2-3 weeks:
After completing the above points, we will start with the implementation of the hidden services.
Can you make a github repo of the torbox.ch website so I can test publishing it? I see it is wordpress and they use cloudflare (curl --head https://torbox.ch
).
Note that publishing a website is just one of the utilities one can do with onion domain. I use for instant messaging communication with XMPP an onion domain.
Note that implementing a website with or without client auth won't differ much, few clicks with the menu or some arguments with the CLI, which is way fast than doing manually.
My timetable is difficult to estimate,
auth server list
when there is not client in there. This is not an error per se, it just does not output anything meaningful saying there is not client, and I want that to be very clear to the user.listen
parameter. I havent started this yet. (Apache maybe if someone send me a apache configuration sample). Nginx resources from MTigas ProPublica with Unix socket and a Single config for Tor and plainnet and Raspiblitz TCP socket with Separate config for Tor and localIf one just want to publish a website via terminal, there is onionshare-cli, I like what they do but I am not sure it fits TorBox because it saves all information to a single chosen file, so they do not modify local torrc and HiddenServiceDir. Anyway, onionshare-cli is really helpful for setting a website to chat, send and receive files, publish a webpage and can add client auth on top of each previous action.
Can you make a github repo of the torbox.ch website so I can test publishing it? I see it is wordpress and they use cloudflare (
curl --head https://torbox.ch
).
Not at the moment. Just use a "Hello world" index.html, the README.md or the static index.htm from https://torbox.ch for testing. At the end, the of a publication of a public website shouldn't orient it on https://www.torbox.ch, but just work for a usual html/php website.
Note that publishing a website is just one of the utilities one can do with onion domain. I use for instant messaging communication with XMPP an onion domain. That sounds interesting and could be another longterm project. OnionShare is a good example what is possible.
Ok, got it working with one virtual port, the 80. For virtual port 443 with https on onion, will implement later.
Save the website files inside /var/www/
. As an example, the folder will be called torbox.ch
. The service will also be called torbox.ch
to be easy to remember.
sudo mkdir -p /var/www/torbox.ch
Create the onion service with unix
or tcp
socket:
SYNTAX: on SOCKET SERVICE VIRTPORT
sh onionservice-cli on unix torbox.ch 80
Activate the web server (nginx in this case):
Note the FOLDER="torbox.ch", no need to write /var/www/torbox.ch
cause the script expect to be in this default folder (/var/www) already.
SYNTAX: nginx STATYS SERVICE FOLDER_INSIDE_VAR_WWW
sh onionservice-cli nginx on torbox.ch torbox.ch
Done, that is all you need to activate an onion service and spin up the web server.
Now get the address that was printed from step 2 and open on Tor Browser. :+1:
If you want to test client auth, you need to configure at least one client: SYNTAX: auth HOST STATUS SERVICE
sh onionservice-cli auth server on torbox.ch alice
The private key will be printed to the terminal, reload Tor Browser and use the key to access the website.
Notes:
Ok, now it is available with nginx or apache2, you can only choose one of course.
Also, menu option to activate or deactivate serving a web site (not the hidden service configuration, the web server configuration for that hidden service)
set the web server in .onionrc
:
Setup the environment with sh setup.sh
Call from any folder the onionservice-tui
:
Now copy from the terminal the address selecting the url and Ctrl+Shit+C
. Paste on Tor Browser and have fun:
pardon my html/css page, I am still learning how to do it.
That looks really great!
But now you want just a you and select clients to access that page, add client auth on your server:
Lets generate a key pair a this is an example, but if the client gave you only his pub key, select the option below of course:
Lets add alice and bob, possible combinations for clients are: alice bob
, alice,bob
, alice, bob`. Comma or space separated.
Important line for Tor Browser is CLIENT_PRIV_KEY=KABLUFI3AMAY62OPOZL3PFHXKMFRE5XCBXBDVLHHAA5V7RNEWZWA
Important line for tor (daemon) is
CLIENT_PRIV_KEY_CONFIG=sjyyyi6y3hqszp4myx3ytghdjpe6gavldymkbwj62wvhsxxs2lg7njqd:descriptor:x25519:KABLUFI3AMAY62OPOZL3PFHXKMFRE5XCBXBDVLHHAA5V7RNEWZWA
Lets copy the browser key option. Reload the Browser page and try again
Now it is authenticated, you need a key to access it.
Try leaving the key field blank or with wrong credentials:
Unfortunately, it will take me a little longer to integrate it into TorBox. My job (the one that pays my bills) is putting me under a bit of workload again.
No problem, with that I have more time to test and to find a job :) The warning on the readme to not trust the repo is because it was only reviewed by myself and developers can be blind on the code they wrote. Also the backup technique needs to be improved, it does its job but I will always want a better backup to be safe.
About you integrating into TorBox, note that you just need to clone and follow the steps on the readme, maybe add one option on the menu to redirect to onion services menu. There is no secret.
But if you really decide to go deep and integrate each option your way, that is okay but I tried to integrate TorBox to Raspiblitz and it worked the first time, but with each new commit and release of TorBox, it becomes harder to keep upgrading individually the files I've changed. The same goes for integrating OnionService to TorBox. (The end result is that I am still building a custom script to change hardcoded paths of TorBox with sed and try to not break things when bulking with sed, which is difficult when dealing with multiple files).
About the long information first comment https://github.com/radio24/TorBox/issues/80#issue-958601443
I have completed all it is possible to do scripting for an onion service. The rest is OpSec that depends on the operator such as:
different machine: consider running the onion service on a different machine (real or virtual) than the actual service. This has the advantage that you can isolate the service from the onion service (a compromise of one doesn’t compromise the other) and helps with isolating potential information leaks isolation: similarly to the above, you can also isolate Tor and the service so it will run on a different network namespace than the service. Tails uses a Tor-or-fail packet filter.
OpSec recommendation to use unix domain as the HiddenServicePort VIRTPORT TARGET is done already for example.
I didn't decide yet on which approach I will integrate onionservice-cli and onionservice-tui into TorBox. At first sight, it seems that I can take onionservice-cli as it is but have to (re-)write my version of onionservice-tui.
One reason I will not only copy your work without understanding the code is that I will also try to contribute to your work. I already did a fork of your project, and if I find something I can contribute, I will make a pull request.
Anyway, great work!
Feel honored, thanks for the fork. Having any questions about why
I did something just say it, because the documentation is more about what the result
of a combination of command will be, instead of why
and how
it is doing that.
Also the TUI problem is the color (that you can set with .dialogrc) or because I use dialog
instead of whiptail
?
I am asking this because I tried to make the most configurable possible so anyone that wants to use it won't have trouble modifying the scripts, if you explain what is lacking I can do enhance.
What are the advantages of dialog
over whiptail
?
FreeBSD's dialog man (read whiptail section in there also)
dselect
and fselect
with dialog, which I can delect directories and files.green dialogrc to fit torbox
# Run-time configuration file for dialog
#
# Types of values:
#
# Number - <number>
# String - "string"
# Boolean - <ON|OFF>
# Attribute - (foreground,background,highlight?,underline?,reverse?)
#
#
# Colors:
#
# Interpret embedded "\Z" sequences in the dialog text by the following character,
# which tells dialog to set colors or video attributes: 0 through 7 are the ANSI
# used in curses: black, red, green, yellow, blue, magenta, cyan and white
# respectively. Bold is set by 'b', reset by 'B'. Reverse is set by 'r', reset by 'R'.
# Underline is set by 'u', reset by 'U'. The settings are cumulative, e.g., "\Zb\Z1"
# makes the following text bold (perhaps bright) red. Restore normal settings with "\Zn".
#
# 0 = black
# 1 = red
# 2 = green
# 3 = yellow
# 4 = blue
# 5 = magenta
# 6 = cyan
# 7 = white
#
# b = bold (set)
# B = bold (unset)
# r = reverse (set)
# R = reverse (unset)
# u = underline (set)
# U = underline (unset)
# n = default (restore)
#
# Source: https://github.com/openoms/joininbox/blob/v0.6.0/scripts/.dialogrc
# Set aspect-ration.
aspect = 0
# Set separator (for multiple widgets output).
separate_widget = ""
# Set tab-length (for textbox tab-conversion).
tab_len = 0
# Make tab-traversal for checklist, etc., include the list.
visit_items = OFF
# Shadow dialog boxes? This also turns on color.
use_shadow = OFF
# Turn color support ON or OFF
use_colors = ON
# Screen color
screen_color = (WHITE,GREEN,ON)
# Shadow color
shadow_color = (BLACK,BLACK,ON)
# Dialog box color
dialog_color = (BLACK,WHITE,ON)
# Dialog box title color
title_color = (BLACK,WHITE,ON)
# Dialog box border color
border_color = (BLACK,WHITE,ON)
# Active button color
button_active_color = (WHITE,GREEN,ON)
# Inactive button color
button_inactive_color = dialog_color
# Active button key color
button_key_active_color = button_active_color
# Inactive button key color
button_key_inactive_color = (GREEN,WHITE,OFF)
# Active button label color
button_label_active_color = (WHITE,GREEN,ON)
# Inactive button label color
button_label_inactive_color = (GREEN,WHITE,ON)
# Input box color
inputbox_color = dialog_color
# Input box border color
inputbox_border_color = dialog_color
# Search box color
searchbox_color = dialog_color
# Search box title color
searchbox_title_color = title_color
# Search box border color
searchbox_border_color = border_color
# File position indicator color
position_indicator_color = title_color
# Menu box color
menubox_color = dialog_color
# Menu box border color
menubox_border_color = border_color
# Item color
item_color = dialog_color
# Selected item color
item_selected_color = button_active_color
# Tag color
tag_color = title_color
# Selected tag color
tag_selected_color = button_label_active_color
# Tag key color
tag_key_color = button_key_inactive_color
# Selected tag key color
tag_key_selected_color = (RED,GREEN,ON)
# Check box color
check_color = dialog_color
# Selected check box color
check_selected_color = button_active_color
# Up arrow color
uarrow_color = (GREEN,BLACK,ON)
# Down arrow color
darrow_color = uarrow_color
# Item help-text color
itemhelp_color = (BLACK,WHITE,OFF)
# Active form text color
form_active_text_color = button_active_color
# Form text color
form_text_color = (BLACK,WHITE,ON)
# Readonly form item color
form_item_readonly_color = (GREEN,BLACK,ON)
# Dialog box gauge color
gauge_color = title_color
# Dialog box border2 color
border2_color = dialog_color
# Input box border2 color
inputbox_border2_color = dialog_color
# Search box border2 color
searchbox_border2_color = dialog_color
# Menu box border2 color
menubox_border2_color = dialog_color
I studied the code of the "onionservice" scripts, and I'm starting to implement some of the features in TorBox v.0.4.3.
"onionservice" gives users the possibility to use "tcp socket" or "UNIX socket". Because "UNIX socket" is the more secure way, I ask myself if I should use this selection as default in TorBox. Are there some disadvantages if "UNIX socket" is used instead of "tcp socket"?
I have nothing to complain about unix socket.
git pull my main branch, I fixed a bug that was implemented here https://github.com/radio24/onionservice/commit/85f95ba6961c01eeb9a1352dc698947c2ac9f20e, fix here https://github.com/nyxnor/onionservice/commit/3958e6d603613fdfe374bad53e3e6c45ad0b61cc
@nyxnor How should we treat this thread?
discussions, there will be always something new to add and other people to report their experiences
My old mod: https://github.com/nyxnor/raspiblitz/blob/tor-patch/home.admin/config.scripts/tor.onion-service.sh New mod: https://github.com/nyxnor/CLI-onion-services/blob/main/tor.onion-service.sh
Currently, the script CREATES, DELETES (optionally purge to delete the onion address), ADD and REMOVE auth, see CREDENTIALS. My script configure client auth server side, but the client side needs to be configure manually. I explained the commands deeply when running the script with echo, but it should be shortened to only display the string necessary and a guide in torbox page or text/ folder explaining how to do so.
This post was edited for reference. This is informational material of things to be implemented and reviewed if were. This is more a long term goal than a complete todo list. The texts were extracted from each link mentioned above them, only the most useful information.
Instructions
Matt Traudt HS Setup
TPO setup onion service
(Optional) Step 5: Running multiple onion services
If you want to forward multiple virtual ports for a single onion service, just add more HiddenServicePort lines. If you want to run multiple onion services from the same Tor client, just add another HiddenServiceDir line. All the following HiddenServicePort lines refer to this HiddenServiceDir line, until you add another HiddenServiceDir line:
Or in Apache with Tor service listening on port 80:
Try to run Tor more securely via a syscall sandbox.
https://www.torproject.org/docs/tor-manual.html.en#Sandbox
Sandbox 1
Disable the SOCKS port. Not like anything else on this box is using tor.
SocksPort 0
Set up the hidden service. propub3r6espa33w.onion -> www.propublica.org
We're using unix sockets instead of "127.0.0.1:xxxxx". see nginx conf.
Docs: https://www.torproject.org/docs/tor-manual.html.en#HiddenServicePort
HiddenServiceDir /var/run/tor/pp_www_hidserv HiddenServicePort 80 unix:/var/run/nginx-pponion-80.sock HiddenServicePort 443 unix:/var/run/nginx-pponion-443.sock
/etc/nginx/sites-enabled/propubonion.conf
#
Note that all of our hostnames listen to a unix socket instead
of "127.0.0.1:xxxxx".
Docs: http://nginx.org/en/docs/http/ngx_http_core_module.html#listen
map $http_upgrade $connection_upgrade { default "upgrade"; "" ""; }
HTTP BARE ONION
server { listen unix:/var/run/nginx-pponion-80.sock; server_name propub3r6espa33w.onion;
allow 127.0.0.1;
}
HTTPS BARE ONION
server { listen unix:/var/run/nginx-pponion-443.sock ssl spdy; server_name propub3r6espa33w.onion;
allow 127.0.0.1;
}
HiddenServiceDir /var/lib/tor/hs-my-website/ HiddenServiceVersion 3 HiddenServicePort 80 unix:/var/run/tor-hs-my-website.sock
add_header Onion-Location http://.onion$request_uri;
server { listen 80; listen [::]:80;
}
server { listen 443 ssl http2; listen [::]:443 ssl http2;
}
server { listen unix:/var/run/tor-hs-my-website.sock;
}
sudo nginx -t
sudo nginx -s reload
wget --server-response --spider your-website.tld
HiddenServiceDir /var/lib/tor/hidden/ftp/ HiddenServicePort 80 192.168.1.1:81
HiddenServicePort 80 unix:/etc/lighttpd/unix.sock