Allow multiple servers

[clowwindy]

1. Design configuration file format
2. Design switching strategy
3. Code!

[riobard]

Suggestion for the config file

# This is a comment line. 

# The config file format is modeled after INI files used by Git and other
# popular software packages.

# Each section defines a client/server instance. The name of the section
# specifies the name of the client/server instance. All key-value pairs 
# except password will be passed to the up/down scripts as environment
# variables. Instance name will be passed as well.

[foo]
mode=server
server=0.0.0.0
port=1440    # a single port (i.e. concurrency=1)
password=foo_password
interface=tun0
network=10.7.0.0/31     # /31 subnet for point-to-point links
up=/etc/shadowvpn/up.sh
down=/etc/shadowvpn/down.sh

[bar]
mode=client
server=0.0.0.0
port=1540-1640  # a port range (i.e. concurrency=100)
password=bar_password
interface=tun1
network=10.7.0.3/31
up=/etc/shadowvpn/up.sh
down=/etc/shadowvpn/down.sh

We should get rid of the old pidfile and logfile config keys and let the respective init system for each OS worry about managing the ShadowVPN process. Most Linux distros ship with systemd now which allow ShadowVPN to run in foreground without worrying about PID and daemonization. Log should be written to stdout and let the managing system to handle log capture and rotation. OpenWRT is migrating to the new procd system providing similar infrastructure. There's no need to handle daemonization in ShadowVPN anymore.

Question: how will ShadowVPN be structured to support multiple clients/servers? Will it adopt a multiprocess architecture similar to Nginx with a master process to setup tun/ports and each child process responsible for each client/server instance?

[clowwindy]

When switching servers, we don't restart ShadowVPN. And we may use different servers at the same time. This means each server just have different addresses and passwords.

We may have multiple ShadowVPN instances of different configurations at the same time, that's why we have our own daemonization, just like OpenVPN, or nginx. Else we have to use something like supervisord to manage them.

[riobard]

Else we have to use something like supervisord to manage them.

I think that's exactly what I was proposing above. Tools like supervisord, systemd, upstart, procd are the future. We don't have to handle daemonization anymore. Just let it run in the foreground and systemd & friends will take care of the rest.

# cat /etc/systemd/system/shadowvpn.service

[Unit]
Description=ShadowVPN server
After=network.target

[Service]
ExecStart=/usr/bin/shadowvpn -c /etc/shadowvpn/server.conf
KillMode=process
Restart=on-failure

[Install]
WantedBy=multi-user.target

[riobard]

OpenWRT, using the new procd init system

# cat /etc/init.d/shadowvpn

#!/bin/sh /etc/rc.common

START=90
USE_PROCD=1

CFG_FILE=/etc/shadowvpn/client.conf

boot()
{
    if [ ! -c "/dev/net/tun" ]; then
        mkdir -p /dev/net
        mknod /dev/net/tun c 10 200
        chmod 0666 /dev/net/tun
    fi
}

start_service() {
    procd_open_instance
    procd_set_param command shadowvpn -c $CFG_FILE
    procd_set_param respawn
    procd_close_instance
}

[clowwindy]

According to records in the Shadowsocks issue tracker, people come and ask how to daemonize the server, again and again. You have to teach them how to do this with Debian, CentOS, Arch, FreeBSD, OS X, OpenWRT, supervisord, Docker, etc. I believe they change more frequently than Shadowsocks itself!

For many years I didn't find anyone who's interested in maintaining these init scripts for different systems. Some of them may sent a pull request, but eventually they will lose interest. So I had the daemonization builtin and got rid of all of the problems.

Anyway, both Shadowsocks and ShadowVPN still support running in the foreground. You can still use procd or supervisord. I think maintaining these scripts are not the main objective of this project. And files under samples are just samples!

[riobard]

I see. Anyway, I was just hoping to simplify the codebase to get rid of unnecessary functionalities (in this case it's the daemonization handling) and delegate to other utilities better suited for the job. If there are enough noobs asking for init scripts and insufficient maintenance of those, leaving current daemonization handling intact is perfectly fine.

We should probably revisit this after systemd has finished taking over the world (if it hasn't already :), at which time we need to support only a handful of init systems: systemd for most Linux distros, procd for OpenWRT, launchd for OS X, and that's pretty much all about it (what do BSDs use?).

Meanwhile, the proposed config file needs to be slightly amended to include pidfile and logfile. Below is a new version

# This is a comment line. 

# The config file format is modeled after INI files used by Git and other
# popular software packages.

# The first non-sectioned area is for general config.

# The following two config are needed when ShadowVPN runs in daemon
# mode (with the -s option). 
pidfile=/var/run/shadowvpn.pid
logfile=/var/log/shadowvpn.log

# Each named section defines a client/server instance. The name of the section
# specifies the name of the client/server instance. All key-value pairs 
# except password will be passed to the up/down scripts as environment
# variables. Instance name will be passed as well.

[foo]
mode=server
server=0.0.0.0
port=1440    # a single port (i.e. concurrency=1)
password=foo_password
interface=tun0
network=10.7.0.0/31     # /31 subnet for point-to-point links
up=/etc/shadowvpn/up.sh
down=/etc/shadowvpn/down.sh

[bar]
mode=client
server=0.0.0.0
port=1540-1640  # a port range (i.e. concurrency=100)
password=bar_password
interface=tun1
network=10.7.0.3/31
up=/etc/shadowvpn/up.sh
down=/etc/shadowvpn/down.sh

[clowwindy]

Since daemon related code is only in daemon.c, and the API is very simple and standard, I think currently it's not a problem.

A common VPN software works with only one tun device, else you have to maintain a complicated routing table. Therefore in client configuration different servers share the keys of interface, network, up, down, too.

interface=tun0
network=10.7.0.0/31     # /31 subnet for point-to-point links
up=/etc/shadowvpn/up.sh
down=/etc/shadowvpn/down.sh
pidfile=/var/run/shadowvpn.pid
logfile=/var/log/shadowvpn.log
concurrency=5

[foo]
mode=server
server=0.0.0.0
port=1440
password=foo_password

[bar]
mode=client
server=0.0.0.0
port=1540-1640
password=bar_password

Concurrency is the number of source ports that the client uses at the same time. It has nothing to do with how many servers we have. With one source port we can have multiple servers (remember this is UDP).

To implement server switching, we need to add some new configurations:

• Switching strategy to use
• Minimal interval to switch to a new server
• How long after which a server is considered dead
• We may cache network statistics somewhere
• And so on

For server configuration, it doesn't have to worry about server switching, so we don't have to change its configuration format.

[riobard]

How do we deal with different subnets used on multiple servers using a single network config key?

[clowwindy]

They should have the same configuration.

[riobard]

I'm not entirely sure about this. Does it mean that I need control over all servers in order to allocate the same subnet? It's a very restrictive limitation. Imagine a customer paying for two ShadowVPN service providers. It's unlikely that the two providers will have the same subnet available for the customer. Provider A might have already allocated 10.7.0.0/31 to another customer even if provider B still has it available. Private IP ranges suck at dealing with allocation conflicts.

But yes, having separate subnets will make routing unnecessarily complicated…

[clowwindy]

Well, this project is designed for hackers, not novices. So it doesn't fit commercial business model. It will remain peer-to-peer mode, which is more efficient, more anonymous.

This task is more like a todo list, I think. My next goal is to support iOS 9. I'll come back to this later.

[riobard]

I see. I originally thought it would follow the shadowsocks model.

[clowwindy]

We don't need another Shadowsocks, if there's already one.