Fixed OpenVPN related sensors (#264)

[jesmak]

Fixed the ID check in get_openvpn function. Did some debugging and it seems that the id-values retrieved from /api/openvpn/service/searchSessions are integers when the VPN-connection is not active. However, when the connection is live, the id-value is in format "ID_IP-ADDRESS:PORT". Modified the id check to work for both scenarios. Also enabled/fixed the connected client count by counting sessions that have a value in connected_since field.

[alexdelprete]

Thanks, LGTM. But @Snuffy2 has the final word.

[Snuffy2]

Thanks @jesmak! Are you using the "new" OpenVPN setup or the legacy one? I'll need to look later, but I believe the new one uses much longer vpnids which are strings and not integers. We may need to account for those as well.

https://docs.opnsense.org/manual/vpnet.html#id3

[jesmak]

Thanks @jesmak! Are you using the "new" OpenVPN setup or the legacy one?

It's the legacy one. It's been a few years since I set it up and it was somewhat complicated, so I don't want to mess with it until I absolutely have to.

[Snuffy2]

Nope, nothing you need to change on your setup. I'll stand-up a simple OpenVPN server on my router using the new setup and confirm what is bring returned and then we can determine how we can best handle both the legacy and new setups.

[Snuffy2]

This is from the new OpenVPN /api/openvpn/service/searchSessions:

No connection:

{
    "total": 2,
    "rowCount": 2,
    "current": 1,
    "rows": [
        {
            "status": "wait",
            "timestamp": 1729039023,
            "virtual_address": "",
            "real_address": "",
            "bytes_received": "0",
            "bytes_sent": "70",
            "type": "client",
            "id": "fc64283f-dbfe-4195-961f-df6bc682918d",
            "description": "Snuffy2's iPhone 16",
            "connected_since": "2024-10-15 20:37:03"
        },
        {
            "status": "ok",
            "type": "server",
            "id": "a90bf93b-0c67-4afa-9beb-5a96c6a956ee",
            "description": "Snuffy2 OpenVPN",
            "timestamp": null,
            "virtual_address": null,
            "real_address": null,
            "bytes_received": null,
            "bytes_sent": null,
            "connected_since": null
        }
    ]
}

With an active connection:

{
    "total": 2,
    "rowCount": 2,
    "current": 1,
    "rows": [
        {
            "status": "wait",
            "timestamp": 1729038711,
            "virtual_address": "",
            "real_address": "",
            "bytes_received": "0",
            "bytes_sent": "42",
            "type": "client",
            "id": "fc64283f-dbfe-4195-961f-df6bc682918d",
            "description": "Snuffy2's iPhone 16",
            "connected_since": "2024-10-15 20:31:51",
            "common_name": null,
            "virtual_ipv6_address": null,
            "connected_since__time_t_": null,
            "username": null,
            "client_id": null,
            "peer_id": null,
            "data_channel_cipher": null,
            "is_client": null
        },
        {
            "status": "ok",
            "type": "server",
            "id": "a90bf93b-0c67-4afa-9beb-5a96c6a956ee_555.555.555.555:64529",
            "description": "Snuffy2 OpenVPN",
            "common_name": "snuffy2-ca",
            "real_address": "555.555.555.555:64529",
            "virtual_address": "10.100.4.2",
            "virtual_ipv6_address": "",
            "bytes_received": "5280",
            "bytes_sent": "5144",
            "connected_since": "2024-10-15 20:31:30",
            "connected_since__time_t_": "1729038690",
            "username": "UNDEF",
            "client_id": "6",
            "peer_id": "1",
            "data_channel_cipher": "AES-256-GCM",
            "is_client": true,
            "timestamp": null
        }
    ]
}

[jesmak]

Ok so it's the same, but number id has been replaced with a guid. I can make a small adjustment to my PR to also cover this case.

[jesmak]

I updated the id comparison, but now I'm wondering if the connected client count also needs some changes. In the session data you pasted, connected_since has a value even when there's no active connection. Maybe we should also check if connection status is "ok" in addition to the connected_since field?

[alexdelprete]

Maybe we should also check if connection status is "ok" in addition to the connected_since field?

it's a good idea, but I also noticed that in the "active connection" json, for "type" client, the connection status is always "wait". So how do we distinguish if it's active or not?

@Snuffy2 are you sure those json outputs are correct? Am I missing something?

[jesmak]

Maybe we should also check if connection status is "ok" in addition to the connected_since field?

it's a good idea, but I also noticed that in the "active connection" json, for "type" client, the connection status is always "wait". So how do we distinguish if it's active or not?

@Snuffy2 are you sure those json outputs are correct? Am I missing something?

I might be wrong, but I was under the impression that in the second json only one connection is active. The one that had its' id change to contain an IP-address.

[alexdelprete]

I might be wrong, but I was under the impression that in the second json only one connection is active. The one that had its' id change to contain an IP-address.

I was looking at "type": "client". But I might be interpreting things incorrectly.

If you notice the client (the iphone), from active json and inactive json, the status is always "wait". Probably the logic is different, but it's not very intuitive to me.

[jesmak]

I was looking at "type": "client". But I might be interpreting things incorrectly.

If you notice the client (the iphone), from active json and inactive json, the status is always ok. Probably the logic is different, but it's not very intuitive to me.

Yeah, it certainly is not intuitive. Here's the session info from my legacy OpenVPN setup. First copy-paste has an active site-to-site VPN connection (id 1) and an inactive road warrior configuration (id 2). Second copy-paste has an active site-to-site VPN connection (id 1) and an active road warrior VPN connection from my Android phone (id 2).

{
    "total": 2,
    "rowCount": 2,
    "current": 1,
    "rows": [
        {
            "status": "ok",
            "type": "server",
            "id": "1_IP_ADDRESS:57696",
            "description": "OBFUSCATED",
            "common_name": "OBFUSCATED",
            "real_address": "IP_ADDRESS:57696",
            "virtual_address": "IP_ADDRESS",
            "virtual_ipv6_address": "",
            "bytes_received": "118407325586",
            "bytes_sent": "4310554814",
            "connected_since": "2024-10-13 11:55:25",
            "connected_since__time_t_": "1728809725",
            "username": "OBFUSCATED",
            "client_id": "3",
            "peer_id": "0",
            "data_channel_cipher": "AES-256-GCM",
            "is_client": true
        },
        {
            "status": "ok",
            "type": "server",
            "id": 2,
            "description": "Road warrior",
            "common_name": null,
            "real_address": null,
            "virtual_address": null,
            "virtual_ipv6_address": null,
            "bytes_received": null,
            "bytes_sent": null,
            "connected_since": null,
            "connected_since__time_t_": null,
            "username": null,
            "client_id": null,
            "peer_id": null,
            "data_channel_cipher": null,
            "is_client": null
        }
    ]
}

{
    "total": 2,
    "rowCount": 2,
    "current": 1,
    "rows": [
        {
            "status": "ok",
            "type": "server",
            "id": "1_IP_ADDRESS:57696",
            "description": "OBFUSCATED",
            "common_name": "OBFUSCATED",
            "real_address": "IP_ADDRESS:57696",
            "virtual_address": "IP_ADDRESS",
            "virtual_ipv6_address": "",
            "bytes_received": "118451730703",
            "bytes_sent": "4312047198",
            "connected_since": "2024-10-13 11:55:25",
            "connected_since__time_t_": "1728809725",
            "username": "OBFUSCATED",
            "client_id": "3",
            "peer_id": "0",
            "data_channel_cipher": "AES-256-GCM",
            "is_client": true
        },
        {
            "status": "ok",
            "type": "server",
            "id": "2_IP_ADDRESS:51503",
            "description": "Road warrior",
            "common_name": "OBFUSCATED",
            "real_address": "IP_ADDRESS:51503",
            "virtual_address": "OBFUSCATED",
            "virtual_ipv6_address": "",
            "bytes_received": "6364",
            "bytes_sent": "5953",
            "connected_since": "2024-10-16 12:43:38",
            "connected_since__time_t_": "1729071818",
            "username": "OBFUSCATED",
            "client_id": "1",
            "peer_id": "0",
            "data_channel_cipher": "AES-256-GCM",
            "is_client": true
        }
    ]
}

As you can see, all of these have the type set as "server". I'm really not sure what this is supposed to mean, as the road warrior connection is definitely initiated by a client app in a mobile device. In addition, there's the "is_client" property which in my examples is always true for active connections. This seems to be how it is in Snuffy2's copy-paste too, assuming "Snuffy2 OpenVPN" in the second json is the only active connection of the four.

So, perhaps the way to calculate active client connections should be: status is "ok", is_client is "true" and connected_since has some value in it.

[jesmak]

I think I've been looking at this a bit wrong. All the time I assumed that Snuffy2 had two different VPN configurations which are shown in the json, but I guess it just shows both the client and server sides separately for a single configuration? The legacy setup I'm using only shows one entry for one configuration. I have to check what is shown in the sessions list when I open two simultaneous road warrior connections from different devices. If a third entry appears, I think it's a safe bet to calculate the connected clients the way I proposed in my previous message.

[jesmak]

Tested with two simultaneous road warrior connections from two different devices and unfortunately, a third entry does not appear. Session list shows the same two entries, with nothing to indicate that there are actually multiple active connections for the other VPN configuration. This makes it quite impossible to count the client connections with the legacy setup. @Snuffy2, do you have another device you could use to test two simultaneous connections with the new OpenVPN setup? If it has the same problem, might make sense to remove the whole sensor from this integration as there's no way to get the correct count for it.

[Snuffy2]

So for the new OpenVPN, Servers and Clients are both an Instance. The server and client I setup are both listed there. Paradoxically, the one with "is_client": true, is the Server and "is_client": null is the Client, although the type field is accurate.

For the client, connected_since always shows a time, even when I'm not connected, so also not accurate.

Let me explore some more.

[travisghansen]

Beware that there are complex configurations with openvpn that can allow the same client (cert, or username, etc depending on config) to be connected multiple times simultaneously.

[Snuffy2]

I'm not able to test 2 concurrent OpenVPN connections right now. @jesmak what does: /api/openvpn/service/searchRoutes show for you? Does it change with 0-2 connections?

[jesmak]

I'm not able to test 2 concurrent OpenVPN connections right now. @jesmak what does: /api/openvpn/service/searchRoutes show for you? Does it change with 0-2 connections?

It does change. Here's the output:

{"total":21,"rowCount":21,"current":1,"rows":[
{"virtual_address":"X.X.X.80C","common_name":"obfuscated","real_address":"XXX.XXX.XXX.XXX:57696","last_ref":"2024-10-17 09:23:12","last_ref__time_t_":"1729146192","type":"server","id":1,"description":"obfuscated"},
{"virtual_address":"X.X.X.81C","common_name":"obfuscated1","real_address":"XXX.XXX.XXX.XXX:57696","last_ref":"2024-10-17 09:23:13","last_ref__time_t_":"1729146193","type":"server","id":1,"description":"obfuscated"},
{"virtual_address":"X.X.X.86C","common_name":"obfuscated","real_address":"XXX.XXX.XXX.XXX:57696","last_ref":"2024-10-17 09:23:12","last_ref__time_t_":"1729146192","type":"server","id":1,"description":"obfuscated"},
{"virtual_address":"X.X.X.87C","common_name":"obfuscated","real_address":"XXX.XXX.XXX.XXX:57696","last_ref":"2024-10-17 09:23:13","last_ref__time_t_":"1729146193","type":"server","id":1,"description":"obfuscated"},
{"virtual_address":"X.X.X.84C","common_name":"obfuscated","real_address":"XXX.XXX.XXX.XXX:57696","last_ref":"2024-10-17 09:23:12","last_ref__time_t_":"1729146192","type":"server","id":1,"description":"obfuscated"},
{"virtual_address":"X.X.X.72C","common_name":"obfuscated","real_address":"XXX.XXX.XXX.XXX:57696","last_ref":"2024-10-17 09:23:05","last_ref__time_t_":"1729146185","type":"server","id":1,"description":"obfuscated"},
{"virtual_address":"X.X.X.70C","common_name":"obfuscated","real_address":"XXX.XXX.XXX.XXX:57696","last_ref":"2024-10-17 09:22:55","last_ref__time_t_":"1729146175","type":"server","id":1,"description":"obfuscated"},
{"virtual_address":"X.X.X.62C","common_name":"obfuscated","real_address":"XXX.XXX.XXX.XXX:57696","last_ref":"2024-10-17 09:22:20","last_ref__time_t_":"1729146140","type":"server","id":1,"description":"obfuscated"},
{"virtual_address":"X.X.X.10C","common_name":"obfuscated","real_address":"XXX.XXX.XXX.XXX:57696","last_ref":"2024-10-17 09:23:03","last_ref__time_t_":"1729146183","type":"server","id":1,"description":"obfuscated"},
{"virtual_address":"X.X.X.85C","common_name":"obfuscated","real_address":"XXX.XXX.XXX.XXX:57696","last_ref":"2024-10-17 09:23:14","last_ref__time_t_":"1729146194","type":"server","id":1,"description":"obfuscated"},
{"virtual_address":"Y.Y.Y.0","common_name":"obfuscated","real_address":"XXX.XXX.XXX.XXX:57696","last_ref":"2024-10-17 09:23:07","last_ref__time_t_":"1729146187","type":"server","id":1,"description":"obfuscated"},
{"virtual_address":"X.X.X.82C","common_name":"obfuscated","real_address":"XXX.XXX.XXX.XXX:57696","last_ref":"2024-10-17 09:23:10","last_ref__time_t_":"1729146190","type":"server","id":1,"description":"obfuscated"},
{"virtual_address":"X.X.X.52C","common_name":"obfuscated","real_address":"XXX.XXX.XXX.XXX:57696","last_ref":"2024-10-17 09:23:15","last_ref__time_t_":"1729146195","type":"server","id":1,"description":"obfuscated"},
{"virtual_address":"X.X.X.83C","common_name":"obfuscated","real_address":"XXX.XXX.XXX.XXX:57696","last_ref":"2024-10-17 09:23:14","last_ref__time_t_":"1729146194","type":"server","id":1,"description":"obfuscated"},
{"virtual_address":"X.X.X.60C","common_name":"obfuscated","real_address":"XXX.XXX.XXX.XXX:57696","last_ref":"2024-10-17 09:22:59","last_ref__time_t_":"1729146179","type":"server","id":1,"description":"obfuscated"},
{"virtual_address":"X.X.X.51C","common_name":"obfuscated","real_address":"XXX.XXX.XXX.XXX:57696","last_ref":"2024-10-17 09:23:15","last_ref__time_t_":"1729146195","type":"server","id":1,"description":"obfuscated"},
{"virtual_address":"X.X.X.63C","common_name":"obfuscated","real_address":"XXX.XXX.XXX.XXX:57696","last_ref":"2024-10-17 09:22:59","last_ref__time_t_":"1729146179","type":"server","id":1,"description":"obfuscated"},
{"virtual_address":"X.X.X.0\/24","common_name":"obfuscated","real_address":"XXX.XXX.XXX.XXX:57696","last_ref":"2024-10-13 11:55:28","last_ref__time_t_":"1728809728","type":"server","id":1,"description":"obfuscated"},
{"virtual_address":"X.X.X.50C","common_name":"obfuscated","real_address":"XXX.XXX.XXX.XXX:57696","last_ref":"2024-10-17 09:23:15","last_ref__time_t_":"1729146195","type":"server","id":1,"description":"obfuscated"},
{"virtual_address":"X.X.X.61C","common_name":"obfuscated","real_address":"XXX.XXX.XXX.XXX:57696","last_ref":"2024-10-17 09:23:06","last_ref__time_t_":"1729146186","type":"server","id":1,"description":"obfuscated"},
{"virtual_address":"Z.Z.Z.6","common_name":"obfuscated","real_address":"YYY.YYY.YYY.YYY:26742","last_ref":"2024-10-17 09:23:06","last_ref__time_t_":"1729146186","type":"server","id":2,"description":"Road warrior"}]}

It seems to show all the devices that connect to my home network via the site-to-site VPN, along with their actual IP-addresses in that remote subnet. These are the rows with id 1.

Also when connecting a device using the road warrior configuration, a new row appears (id 2). However, when connecting another device using the same road warrior configuration, a new row does not appear. The devices themselves show the same private IP on the OpenVPN app, so I guess it makes some sort of sense considering the previous message by @travisghansen. I'm by no means a networking expert but it feels really weird to me that these two separate physical devices connecting from two separate physical networks would be considered the same client, but that is indeed how it seems.

[Snuffy2]

Yes, the Road Warrior scenario is very weird.

You're saying that two separate devices one with a public IP of 111.111.111.111 and another with public IP of 222.222.222.222 when connected to the Road Warrior OpenVPN both get the same OpenVPN IP of 333.333.333.333 (IPs obviously made up)??

Also, which IP shows in the real_address field? That is supposed to the public IP of the client connected to the OpenVPN Server, right?

[jesmak]

Yes, the Road Warrior scenario is very weird.

You're saying that two separate devices one with a public IP of 111.111.111.111 and another with public IP of 222.222.222.222 when connected to the Road Warrior OpenVPN both get the same OpenVPN IP of 333.333.333.333 (IPs obviously made up)??

Also, which IP shows in the real_address field? That is supposed to the public IP of the client connected to the OpenVPN Server, right?

This is exactly how it seems to be, at least in my environment. real_address for the site-to-site VPN seems to have the public IP-address of the remote location connected to my home network. For the road_warrior connections, it's sometimes the public IP of my Android phone's mobile connection (WiFi is disabled), and sometimes it's the (private) IP-address of my laptop which is connected to my WiFi. So it's not static when both devices are connected. Which IP is shown in /services/searchRoutes, it seems to change randomly. And yes, the private IP of the VPN-connection displayed in both clients is the same.

[Snuffy2]

Bizarre. I guess, as you proposed, we should just remove the Connected Client sensor for now. I'll be updating some of that code likely this weekend so I can do that part. I'll be adding in similar data for Wireguard VPNs. Since Wireguard VPNs are stateless, there also won't be a Connected Client count sensor for it either.

If you want to refine your PR to just fix the id issue for now, that would be great.

[jesmak]

Hey! Sorry I was a bit busy so didn't get back to this until now, but I see you already took care of it. Thank you!