OpenVPN / openvpn

OpenVPN is an open source VPN daemon
http://openvpn.net
Other
10.32k stars 2.92k forks source link

iroute sometimes gets lost using dco-module #438

Open mr-mister123 opened 8 months ago

mr-mister123 commented 8 months ago

Describe the bug I have a server and a client previously running debian buster. The client is running in daemon mode and is always connected. i had a stable connection for years. The clients network (192.168.89.0/24) is routed to the server by a) giving it a fixed ip-adress within the vpn-subnet b) and setting an iroute in the client-file within the ccd-dir.

additionally the route-property (192.168.89.0/24) is set in the server.conf.

After upgrading both of them (didn't check what happens if only the server was upgraded) to bookworm using openvpn 2.6.3 and enabling the dco-module, behaviour changed as excepted: since the dco-module is using kernel-routes (instead of the old internal routing-mechanism) i now have one route on the serverside, that is always set, if the server-daemon is running (resulting from the route-property):

192.168.89.0/24 via 192.168.23.2 dev tun0 metric 200

and one route, that is automatically beeing added if the client is connected:

192.168.89.0/24 via 192.168.23.253 dev tun0 metric 100

So far so god...

But: There is a strange behavior that seems to occur, if the client changes its public ip-address, for example through dhcp by the provider (i think, it has to do with that, but i'm not sure).

The route added by the dco-module (because of the iroute-command in client ccd) is getting lost.

ip route show

then doesn't list it anymore. The 192.168.89.0/24-network is then not reachable anymore. But the vpn-channel seems to be alive since i can access the client through its static vpn-subnet-ip (192.168.23.253).

To Reproduce

Expected behavior The route created by the dco-module should not get lost

Version information (please complete the following information): Server:

Client:

Additional context I have an other client still running buster and therefore without dco enabled. this doesn't cause any problems. but it doesn't change its ip-adress. I dont know if it is related to that or if it makes a difference, when the client is using the dco-module. this just as a hint to you...

thanks for your effort and greetz, Karsten

ordex commented 8 months ago

Hi @mr-mister123 and thanks a lot for your report. Any chance you could provide a server log with --verb 6 so we can check what the server is exactly doing and why/when that route is eliminated?

As a side note, do you know where 192.168.23.2 is coming from in the other route?

mr-mister123 commented 8 months ago

Hi,

As a side note, do you know where 192.168.23.2 is coming from in the other route?

This route is beeing added, if 'route' is set in the server.conf. Man-page states:

However, when using DCO, the --iroute directive is usually enough for DCO to fully configure the routing table. The extra --route directive is required only if the expected behaviour is to route the traffic for a specific network to the VPN interface also when the responsible client is not connected (traffic will then be dropped).

That's exactly what i want to achive (drop packets to 192.168.89.0/24 if client is not connected). Already tested without that route-directive, but the route added by dco-module still get's lost sometimes.

Any chance you could provide a server log with --verb 6 so we can check what the server is exactly doing and why/when that route is eliminated?

I've activated verb 6, now i have to wait for the route to get lost. this can take a while. i hope, i'll manage it at all, since i do not monitor that route the whole time... and (somehow crazy) it get's repaired 'by self': it was broken yesterday, but for example now it's fixed although i didn't change anything (neither server nor client-daemon were restarted). Maybe it 'toggles'?!

i think, this is a race-condition. cause of the ip-adress-change the server closes connection. simultaniously the client starts a new connection. on connect the route is still there and no new route will be added. but the close of the old connection is still pending. after that the rout gets removed an now it's missing.. can this be?

ordex commented 8 months ago

it could be related to overlapping events. yes. This is exactly what I am hoping to see in the log, because it' the openvpn daemon that installs/removes routes and the daemon is a running a single thread. So the log should be fairly clear about what is going on.

cron2 commented 8 months ago

Hi,

On Wed, Oct 25, 2023 at 01:12:16AM -0700, Antonio Quartulli wrote:

it could be related to overlapping events. yes. This is exactly what I am hoping to see in the log, because it' the openvpn daemon that installs/removes routes and the daemon is a running a single thread. So te log should be fairly clear about what is going on.

Yeah, something like

this is never a problem without DCO (as iroutes are not formally removed, they just disappear) but with the explicit route removal call things might break.

gert -- Gert Doering - Munich, Germany @.***

mr-mister123 commented 8 months ago

Hi,

finally it happend ;-)

now i have the state, that the route is missing, but connection seems to be active. i can reach the client through its fixed vpn-ip-address 192.168.23.253, but not through the routed 192.168.89.0/24-net...

here is the log:

openvpn.log

the route got lost at 2023-10-28T06:16:08, re-connected at 2023-10-28T07:09:22. but no new route was created... interestingly it's quite a long time-range between route-deletion and reconnection, so the problem doesn't seem to be related to overlapping events... ?

Greetz, Karsten

ordex commented 8 months ago

Thanks. This is indeed interesting. And this is not a disconnection and reconnection, we rather have a rekeying timeout which ends up with OpenVPN "restarting" the client instance (this is when the route is removed). Later (at 07:09:22) the client performes a new renegotiation which works as expected. However, this is not a full reconnection, therefore routes are not added again.

I am leaning towards "upon instance restarting we should not delete the route, because the instance is still there"

ordex commented 8 months ago

Although I don't understand why the client is switching from peer-id 2 to peer-id 1, despite not going through a real reconnection..