Controller: units continuously toggles connected/disconnected

[gsanchietti]

Upon exceeding approximately 20 units, the list of units connected to the controller begins to exhibit anomalous behavior. The status of the VPN connection continuously toggles between connected and disconnected randomly, even though the connection remains up. When a unit is shown as disconnected, the connection link is not displayed, which is appropriate.

Cause

• The API providing the list data may take longer than the UI refresh interval.
• The OpenVPN socket often returns random values.

Steps to reproduce

1. Connect more than 20 units to the controller.
2. Observe the list of connected units in the controller UI.
3. Notice the VPN connection status toggling between connected and disconnected randomly.

Expected behavior

The unit list should remain stable, accurately reflecting the connection status of each unit.

Actual behavior

The unit list shows erratic behavior, with connection statuses toggling randomly.

Proposed Solution

• Investigate API performance and adjust the UI refresh interval accordingly.
• Address issues with the OpenVPN socket returning random values.

Components

NS8 NethSecurity version: 0.0.22

See also

https://github.com/NethServer/nethsecurity/assets/7226896/0157d5d3-de78-478f-b8e7-e5a62ba3566b

[gsanchietti]

Test case 1

• On a NS8 cluster, install the latest controller release: add-module ghcr.io/nethserver/nethsecurity-controller:latest
• Check the issue is not reproducible

Test case 2

• Update the controller application on an existing NS8 cluster:

  api-cli run update-module --data '{"module_url":"ghcr.io/nethserver/nethsecurity-controller:latest","instances" [nethsecurity-controller1"]}'

• Check the issue is not reproducible

[gsanchietti]

Test case 1 failed: a new connected machine, does not have remote info so it's not possible to connect to machine and operate on it.

Failed request:

curl -X GET 'https://controller.gs.nethserver.net/api/units/28289956-e16f-4379-9417-631225cc6478/token' --insecure -H 'Content-Type: application/json' -H 'Authorization: Bearer xxx' 
{"code":400,"data":"","message":"request failed for: 28289956-e16f-4379-9417-631225cc6478"}

Log:

Jul 08 10:54:08 rl1.leader.cluster0.gs.nethserver.net api[238490]: nethsecurity_controller 2024/07/08 10:54:08 middleware.go:181: [INFO][AUTH] authorization success for user admin. GET /units/28289956-e16f-4379-9417-631225cc6478/token 
Jul 08 10:54:10 rl1.leader.cluster0.gs.nethserver.net api[238490]: [GIN] 2024/07/08 - 10:54:10 | 400 |  2.002186352s |       127.0.0.1 | GET      "/units/28289956-e16f-4379-9417-631225cc6478/token"
Jul 08 10:54:10 rl1.leader.cluster0.gs.nethserver.net proxy[238554]: 127.0.0.1 - - [08/Jul/2024:10:54:08 +0000] "GET /api/units/28289956-e16f-4379-9417-631225cc6478/token HTTP/1.1" 400 108 "-" "-" 488 "routerapi@file" "http://127.0.0.1:20021/" 2002ms
Jul 08 10:54:10 rl1.leader.cluster0.gs.nethserver.net traefik[57500]: 5.90.220.13 - - [08/Jul/2024:10:54:08 +0000] "GET /api/units/28289956-e16f-4379-9417-631225cc6478/token HTTP/2.0" 400 91 "-" "-" 4037 "nethsecurity-controller2-https@file" "http://127.0.0.1:20023" 2003ms

[gsanchietti]

Test case 1 verified

[gsanchietti]

Test case 2 verified

[gsanchietti]

Released ns8-nethsecurity-controller 0.0.23