config: accept port 0

[yogo1212]

from the docs:

When an address without port or with port 0 is configured, a new socket with a random port will be created for each outgoing connection.

however: with bind any:0 default;, fastd fails with Error: config error: invalid port. bind any port 0 default; has the expected result - EDIT: goof on my side.

bind any default; should, as far as i understand the docs, also cause fastd to bind to a port, but there's neither an error message nor a listening port (checking with ss) - untouched in this PR.

i'm completely new to fastd, cool stuff.

[neocturne]

Thanks for the PR - but I'd prefer to change the docs, as there is no good reason to allow passing port 0. any:0, any port 0 and any should be completely equivalent in a bind definition with your patch - see the maybe_port rule.

(also :<port> and port <port> are exactly the same - the latter syntax only exists to make remote definitions with hostnames look less weird)

[yogo1212]

i just noticed that port 0 and :0 behave the same - don't know what i tested earlier.

well, i'm interested in having the kernel choose a port automatically for me - like what many other programms allow me to do. what reason would you need?

or rather: why should fastd artificiallly add a restriction to bind for this?

[neocturne]

Your change just makes bind any:0 default; equivalent to bind any default;, the resulting configuration is already possible.

The docs also explain why you don't see a socket with ss: "When an address without port or with port 0 is configured, a new socket with a random port will be created for each outgoing connection." On addresses specified without port, no incoming connections will be accepted, so the sockets are only opened when attempting to open a session with a peer.

[yogo1212]

but that is exactly what i want: spawn a new fastd instance, let them figure out a port, and then send that port to the peers.

in case anyone else wants to do this, here's my workaround using python:

port="$(python3 <<EOF
import socket

s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind(('', 0))
addr = s.getsockname()
print(addr[1])
s.close()
EOF
> )"

or using busybox netcat with NC_SERVER and NC_110_COMPAT: port="$(nc -vluw 1 2>&1 | sed -En 's/.*:(\d+).*/\1/p')"

and then put the port into the config. not perfect though (ipv4/ipv6, python dependency).

[yogo1212]

what is the downside of allowing port 0?

[yogo1212]

sorry, friend. it is too late for me :-( i don't mean to be rude.

if you think that's cannon, i trust you to know better than me what consistent behaviour is for fastd.

i was just irritated that the option called "bind" in fastd restricts the features from "bind" int the socket API in a way i don't understand. taking into account what you said last, i wonder why specifying a port or not changes when bind is called.

[neocturne]

Seems like a very obscure usecase to me, sorry. Either you want fastd to be reachable for incoming connections, in which case a fixed port should be used, or you want outgoing connections only with a random bind port, which is possible already. This is how most "daemon" software which handles incoming connections behaves.

Even if you have a way to find out the randomly selected port and send that to peers, this port will change when fastd (or the host running fastd) is restarted, leading to unnecessary connections drops.

As noted, your proposed change will not achieve what you want - instead, the internal bind address struct would need to be extended with an additional flag to distinguish a fixed random bind and the existing dynamic per-peer binds. This seems like too much complexity for such an unusual usecase to me.

A bit of trivia: During early development (pre-0.4), fastd actually behaved like you expected: fastd would bind to a random port on start and use this socket for all connections. This turned out be be an issue when running behind certain broken NAT routers as they are installed on many home internet lines: They expected UDP to behave like TCP, with each address+port combination talking to only a single remote host, dropping all packets to/from other hosts. For this reason, a separate socket with a random port was introduced for each peer.

This turned out to work better, but there were still even more broken routers with broken PMTU detection which would sometimes get stuck with tiny MTU values. For this reason, the peer sockets are recreated whenever a connection is lost, so the new connection would use a new port with non-broken MTU...

[neocturne]

Seems like a very obscure usecase to me, sorry. Either you want fastd to be reachable for incoming connections, in which case a fixed port should be used, or you want outgoing connections only with a random bind port, which is possible already. This is how most "daemon" software which handles incoming connections behaves.

This part could have been worded better. What I wanted to say is that daemon software usually doesn't allow you to use a random port at all.

[yogo1212]

use case aside: the way i read your explanation and your intention to change the docs, i gather you want to discourage people from creating configurations that don't work behind NATs. another question could be about documenting why the default flag can't be used together with an empty port.

at least for me, there's some confusion because the purpose of socket-api-bind is to "assign an address" (including the port) - which doesn't happen with fastd-bind unless a non-zero port is used. respectively, the empty port is used to encode "random individual ports for outgoing connections" (or rather: per-peer sockets) but that makes it unavailable for use with socket-api-bind where it has a special meaning.

i'm not proposing to add any extra flags to the internal bind struct or any new program logic; i only want bind any default or bind any to actually do a call to socket-api-bind. for me, removing the 0 check in the lexer is enough - unless i'm misunderstanding the way the port is treated further down the line. probably, a more proper way would be to add per-peer and no-per-peer options to fastd-bind. EDIT: or maybe splitting bind into "listen"-bind and peer-bind.

let me know what you think ;-)

btw, GNU nc -l -u or darkhttpd . --port 0 work fine.

the rest is only about the use case - safe to skip: my application is a VPN-server that dynamically creates distinct networks based of an identifier. one fastd instance is spawned per network and the list of networks isn't static. i could use a database field with a unique-constraint to determine free ports in advance but the kernel is more suited to make informed decisions about that.

after authorisation, clients should receive hostname, port, and public key of the fastd instance hosting their respective network. the clients public key is then added to the server config - the connection info is uncertain because of NAT. conceptually, the listening socket needs to be up for clients to be able to connect. there are no dynamic per-peer binds in this kind of setup. if there were any, it would be necessary to extract the port for each peer and send it to them.

the server is not behind a NAT and should have no problem handling many clients on one port. re-using the port of previous instance on restart won't be a problem. and if it was behind a NAT, the determined-in-advance port wouldn't make a difference. the 'default' part (fixed outgoing port) would be used to save ports when connecting multiple relays.

[yogo1212]

another use case for the ephemeral port with a shared socket across connections is udp hole punching. for udp hole punching, NAT is expected to work correctly. of course, shared sockets with pre-determined port also allow hole punching but it's arguably pointless to specify a port that is not meant to be used from anyone outside. plus there's always the risk of collisions.

with fastd unwilling to bind to a ephemeral port on startup and for a clean flow of information inside the application, it is necessary to resort to workarounds (i've added an openwrt-example using netcat above).

[neocturne]

i'm not proposing to add any extra flags to the internal bind struct or any new program logic; i only want bind any default or bind any to actually do a call to socket-api-bind. for me, removing the 0 check in the lexer is enough - unless i'm misunderstanding the way the port is treated further down the line.

This is not sufficient. With your change, any:0 and any port 0 are just treated the same way any already is - see the maybe_port nonterminal rule in config.y, which just sets the port to 0 when none is given. We can't add the additional logic you want without extending the bind address struct.

For UDP hole punching, I don't see how a random port on the fastd side would help as long as fastd can't punch holes itself. If a separate application is necessary to punch the hole, I'd expected fastd to be started with the same port after the hole punching has succeeded - so from the fastd point of view this is simply a fixed port.

A somewhat different approach that might solve multiple issues would be to add an option to accept a bound UDP socket file descriptor from the environment or command line. I've been thinking about adding this to support systemd .socket units for some time - but it could generally allow fastd instance managers to create a socket ahead of time, with a robust way to determine the bind port.

[yogo1212]

This is not sufficient.

Ok. understood, my wrong. I didn't anticipate the logic for deciding when and how many sockets are opened was depending on the port in the bind address struct.

... as long as fastd can't punch holes itself

Sure, fastd can punch holes already:

• fastd peers A and B connect to a server
• the server passes on the peer information it observes from A and B (IP/port) via the side channel
• peer A, using the same socket used to connect to the server, connects to peer B using the information observed by the server
• and vice-versa

One of the first datagrams will drop but that's not an issue. The only necessary "hack" is knowing a port in advance.

I don't see how a random port on the fastd side would help.. .. random port ..

The randomness isn't that relevant, the availability is. Conceptually, each fastd-instance needs at least one port to use for outgoing connections. Without any changes to fastd, I need to get a free port from the operating system each time a new VPN is created. Using a port range and writing port management logic just seems unreasonable if the kernel does that anyway and more reliably.

.. how a random port on the fastd side wide help

Sadly, the code for getting a free port is platform-specific: Tools like ruby or python are great to bind a socket and get the local port but they are usually not installed in openwrt. The cheapest option i have at the moment is requiring a busybox' netcat built with 110_compat- and server-support and do the workaround described above - which is also not great. It's acceptable to require python on CentOS but it's also not pre-installed. The busybox netcat solution for openwrt doesn't work with GNU netcat - which also wouldn't be pre-installed. Installing python just to be able to pass port 0 to bind just seems a bit weird.

Hole punching aside, the same problem occurs to spawning multiple instances of fastd for peers to connect. fastd won't bind without a specific port. You say fastd will open a socket once it has a peer to connect to - but for that a port is needed; The client can't know the port because it also won't bind before it has anything to connect to and also it's most likely going to be behind a NAT, so knowing the local port doesn't help anyway.

The sane way to get an arbitrary and available port is to pass port 0 to bind.

an option to accept a bound UDP socket file descriptor

Imho, passing a listening fd through a unix socket is worse than the current solution for ephemeral port bindings because then i'd need a C application to send the fd with.

support systemd .socket units

Even if systemd sockets support binding on port 0 that solution won't work with openwrt anytime .. ever, probably.

Really, the best option I see at the moment for a solution in fastd is changing the config file format. If that is an option for you or if we can agree on something else, I'd be willing to implement it.

A minimally invasive way could be adding something like the per-peer-socket flag but that requires changing the default interpretation of the 0 port and would thus silently change the meaning of existing configs. The other way around, adding a force-bind-or-so flag, looks like a hack. a listen bind doesn't make sense together with the default flag, which is difficult to check in the lexer without more possible bind expansions.

For fastd, i'm just some random dude on the internet but i will still seriously propose purposfully breaking existing configurations, removing the bind paradigm, and replacing it with two other options, maybe listen and connector.

listen [<IP address>]:<port> [ interface "<interface>" ] [ connector [ ipv4 | ipv6 ] ];
connector [<IP address>][:<port>] [ interface "<interface>" ] [ ipv4 | ipv6 ];

"listen" controls how incoming connections are handled, "connector" controls how outgoing connections are created. "listen" replaces existings binds with a port, "connector" replaces existing binds without a port. Additionally, "listen" can handle port 0 in an idiomatic way. "connector" with a port could allow specifying a single socket for outgoing connections that can't be used for incoming connections.

I know this is crazy, i just honestly think the existing config paradigm has this serious downside.

[neocturne]

... as long as fastd can't punch holes itself

Sure, fastd can punch holes already:

• fastd peers A and B connect to a server
• the server passes on the peer information it observes from A and B (IP/port) via the side channel
• peer A, using the same socket used to connect to the server, connects to peer B using the information observed by the server
• and vice-versa

Pretty hacky, as the server would need to be configured as a peer in fastd, even when it is only supposed to act as a hole-punching counterpart, but I'll admit that this is possible.

The randomness isn't that relevant, the availability is. Conceptually, each fastd-instance needs at least one port to use for outgoing connections. Without any changes to fastd, I need to get a free port from the operating system each time a new VPN is created. Using a port range and writing port management logic just seems unreasonable if the kernel does that anyway and more reliably.

.. how a random port on the fastd side wide help

Hole punching aside, the same problem occurs to spawning multiple instances of fastd for peers to connect. fastd won't bind without a specific port. You say fastd will open a socket once it has a peer to connect to - but for that a port is needed; The client can't know the port because it also won't bind before it has anything to connect to and also it's most likely going to be behind a NAT, so knowing the local port doesn't help anyway.

The "both sides behind NAT" is not something I've seriously considered so far - I'd expect NATted instances to open outgoing connections only, or the admin configuring a port forward in the NAT router (which also requires a fixed port). In all "intended" setups, either both sides use fixed ports, or one side has a fixed port, and the other side opens the connection using a random source port which may change.

The sane way to get an arbitrary and available port is to pass port 0 to bind.

an option to accept a bound UDP socket file descriptor

Imho, passing a listening fd through a unix socket is worse than the current solution for ephemeral port bindings because then i'd need a C application to send the fd with.

systemd doesn't use a UNIX socket for this, it simply execs with the sockets passed in fd 3+, plus environent variables LISTEN_PID and LISTEN_FDS (see https://www.freedesktop.org/software/systemd/man/sd_listen_fds.html). On OpenWrt I'd use Lua with Luaposix to implement this.

support systemd .socket units

Even if systemd sockets support binding on port 0 that solution won't work with openwrt anytime .. ever, probably.

Yes, this is just a different usecase that could be supported by a socket fd passing feature.

Really, the best option I see at the moment for a solution in fastd is changing the config file format. If that is an option for you or if we can agree on something else, I'd be willing to implement it.

A minimally invasive way could be adding something like the per-peer-socket flag but that requires changing the default interpretation of the 0 port and would thus silently change the meaning of existing configs. The other way around, adding a force-bind-or-so flag, looks like a hack. a listen bind doesn't make sense together with the default flag, which is difficult to check in the lexer without more possible bind expansions.

As port 0 is disallowed at the moment, we can use that without introducing any new config flags (it will require some changes to internal data structures though). I'm not completely opposed to this anymore, mostly because of the hole punching scenario (although I still believe there are better solutions).

For a fastd instance manager, I still believe this is a bad idea, as you'd have to use ss/netstat/lsof (or their underlying mechanisms Netlink/procfs) to find the port selected by the kernel, or parse fastd log message, both of which is extremely ugly IMO (unless you have a different approach that I didn't think of?).

I'll have a look at the required changes for a static "port 0" bind to judge whether I consider the added complexity acceptable.

[neocturne]

I've pushed a branch fixed-random-port which implements this feature, please test.

[yogo1212]

Pretty hacky, as the server would need to be configured as a peer in fastd, even when it is only supposed to act as a hole-punching counterpart, but I'll admit that this is possible.

When using batman, the server can be used as a backup relay ;-)

systemd doesn't use a UNIX socket for this, it simply execs with the sockets passed in fd 3+, plus environent variables LISTEN_PID and LISTEN_FDS.

I got it mixed up with the status socket. Thanks for clearing it up for me :-)

you'd have to use ss/netstat/lsof (or their underlying mechanisms Netlink/procfs) to find the port selected by the kernel, or parse fastd log message, both of which is extremely ugly IMO

You're right :-( This is what I'm doing atm: ss -tulpn | grep -E "pid=$fastd_pid\>" | sed -E 's/\s+/ /g' | cut -f5 -d' ' | cut -f2 -d':' For the openwrt-server the same thing works with netstat... It's far from glorious :-/

But the idea with procfs is really neat! So, thanks again :+1:

I've pushed a branch fixed-random-port which implements this feature, please test.

I will! Should be done by Tuesday.

[yogo1212]

From my end, it works :cake:

So just to be sure: bind any default; doesn't bind on startup, bind any port 0 default; does. No port is the per-peer setting, any other port causes an immediate bind, and this config change is backwards-compatible because existing configs with explicit port 0 were forbidden?

I like it :-) Thanks for the work!