Establish hook not always called on connect

mweinelt commented 2 years ago

Every day the offloader in our hackspace reconnects to a gateway and the link ends up down, because the establish hook is not called. This connection always uses null@l2tp and we are using a multitap setup.

May 05 12:23:30 gw02.darmstadt.freifunk.net fastd@dom1_1312[4775]: received handshake from <64283-darmstadt.ccc.de>[[hidden]:56205]
May 05 12:23:30 gw02.darmstadt.freifunk.net fastd@dom1_1312[4775]: responding handshake with <64283-darmstadt.ccc.de>[[hidden]:56205]...
May 05 12:23:30 gw02.darmstadt.freifunk.net fastd@dom1_1312[4775]: received handshake finish from <64283-darmstadt.ccc.de>[[hidden]:56205] using fastd v22
May 05 12:23:30 gw02.darmstadt.freifunk.net fastd@dom1_1312[4775]: handling handshake finish with <64283-darmstadt.ccc.de>[[hidden]:56205]...
May 05 12:23:30 gw02.darmstadt.freifunk.net fastd@dom1_1312[4775]: [hidden]:56205 authenticated as <64283-darmstadt.ccc.de>
May 05 12:23:30 gw02.darmstadt.freifunk.net fastd@dom1_1312[4775]: initializing L2TP offload device...
May 05 12:23:30 gw02.darmstadt.freifunk.net fastd@dom1_1312[4775]: L2TP offload device `peer-5227d2430a' initialized.
May 05 12:23:30 gw02.darmstadt.freifunk.net fastd@dom1_1312[4775]: new session with <64283-darmstadt.ccc.de> established using method `null@l2tp'.
May 05 12:23:30 gw02.darmstadt.freifunk.net fastd@dom1_1312[4775]: invalidating old session with <64283-darmstadt.ccc.de>
May 05 12:23:30 gw02.darmstadt.freifunk.net fastd@dom1_1312[4775]: cleaning left handshakes with <64283-darmstadt.ccc.de>
May 05 12:23:30 gw02.darmstadt.freifunk.net fastd@dom1_1312[4775]: received packet from offloaded session

The configuration roughly looks like this:

interface "peer-%k";
persist interface no;
mtu 1312;
mode multitap;
offload l2tp yes;

on establish async "/etc/fastd/hooks/establish dom1-peers";
on disestablish async "/etc/fastd/hooks/disestablish";

secret "...";

method "null@l2tp";
method "null";
method "salsa2012+umac";

and a successful connect looks like so:

May 05 19:11:21 gw05.darmstadt.freifunk.net fastd@dom1_1312[3144]: generating new handshake key
May 05 19:11:21 gw05.darmstadt.freifunk.net fastd@dom1_1312[3144]: received handshake from <64283-darmstadt.ccc.de>[[hidden]:35872]
May 05 19:11:21 gw05.darmstadt.freifunk.net fastd@dom1_1312[3144]: responding handshake with <64283-darmstadt.ccc.de>[[hidden]:35872]...
May 05 19:11:21 gw05.darmstadt.freifunk.net fastd@dom1_1312[3144]: received handshake finish from <64283-darmstadt.ccc.de>[[hidden]:35872] using fastd v22
May 05 19:11:21 gw05.darmstadt.freifunk.net fastd@dom1_1312[3144]: handling handshake finish with <64283-darmstadt.ccc.de>[[hidden]:35872]...
May 05 19:11:21 gw05.darmstadt.freifunk.net fastd@dom1_1312[3144]: [hidden]:35872 authenticated as <64283-darmstadt.ccc.de>
May 05 19:11:21 gw05.darmstadt.freifunk.net fastd@dom1_1312[3144]: initializing L2TP offload device...
May 05 19:11:21 gw05.darmstadt.freifunk.net fastd@dom1_1312[3144]: L2TP offload device `peer-5227d2430a' initialized.
May 05 19:11:21 gw05.darmstadt.freifunk.net fastd@dom1_1312[3144]: connection with <64283-darmstadt.ccc.de> established.
May 05 19:11:21 gw05.darmstadt.freifunk.net fastd@dom1_1312[3144]: new session with <64283-darmstadt.ccc.de> established using method `null@l2tp'.
May 05 19:11:21 gw05.darmstadt.freifunk.net fastd@dom1_1312[3144]: cleaning left handshakes with <64283-darmstadt.ccc.de>
May 05 19:11:21 gw05.darmstadt.freifunk.net fastd[437330]: peer-5227d2430a::establish: Waiting for interface
May 05 19:11:21 gw05.darmstadt.freifunk.net fastd[437330]: peer-5227d2430a::establish: Disabling accept_ra
May 05 19:11:21 gw05.darmstadt.freifunk.net fastd[437330]: peer-5227d2430a::establish: Setting MTU to 1312
May 05 19:11:21 gw05.darmstadt.freifunk.net fastd[437330]: peer-5227d2430a::establish: Waiting for bridge
May 05 19:11:21 gw05.darmstadt.freifunk.net fastd[437330]: peer-5227d2430a::establish: Connecting to bridge
May 05 19:11:21 gw05.darmstadt.freifunk.net fastd[437330]: peer-5227d2430a::establish: Isolating bridge port
May 05 19:11:21 gw05.darmstadt.freifunk.net fastd[437330]: peer-5227d2430a::establish: Pulling link up
May 05 19:11:21 gw05.darmstadt.freifunk.net fastd@dom1_1312[3144]: child process 437328 finished

neocturne commented 2 years ago

The on-establish hook is not run when renewing an existing connection; this is expected. Unfortunately is it sometimes necessary to tear down and recreate L2TP offload interfaces even for a renewal, as some settings can't be changed for existing L2TP interfaces.

For interface setup, use on-up, which is called each time an interface is created (and for each peer in TUN and Multi-TAP mode).

Note that on-up is currently not passed all environment variables that on-establish gets (in particular, no IP address and port information). This is unlikely to be an issue in practice, as this information is fairly useless with on-establish as well for anything but logging purposes, as addresses and ports may change without on-establish being called again...

mweinelt commented 2 years ago

What would be the appropriate hook to clean up after disconnect/timeout? Thinking of setting link down, nomaster and removing the link, or is that entirely pointless?

The documentation on when these hooks are called is slightly unintuitive.

neocturne commented 2 years ago

Hmm yes, I think some of the documentation predates the multi-interface modes of fastd and could use some updates and clarification...

The rules are:

on-up is run just after the interface has been created, on-down just before the interface is removed
In TUN and Multi-TAP mode, on-establish is normally run after on-up, and on-disestablish is run before on-down. However, additional on-down+on-up pairs can happen on renewal without further on-establish/on-disestablish when L2TP offloading is enabled - either because a new L2TP interface needs to be created to change tunnel configuration, or when switching between null@l2tp and another method.
Each on-up call is eventually matched by an on-down and each on-establish by an on-disestablish
With persist interface no, a peer interface is removed as soon as its connection times out (or is disestablished for another reason)

on-down is usually the right place for cleanup, if there is anything that isn't cleaned up simply by the interface disappearing.

neocturne commented 2 years ago

Maybe it would be a good idea to deprecate on-establish/on-disestablish in TUN and Multi-TAP modes to reduce confusion, and possibly extend peer-specific on-up/on-down to include the same environment as on-establish/on-disestablish currently do.

Edit: This only makes sense when persist interface no is also set

neocturne / fastd

Establish hook not always called on connect #26