search

NethServer / dev

NethServer issue tracker
https://github.com/NethServer/dev/issues
63 stars 18 forks source link

Statistics on OpenVPN connections #5827

Closed andre8244 closed 5 years ago

andre8244 commented 5 years ago

At the moment the available information about OpenVPN connections consists of the current number of active connections. This piece of data is displayed on the Cockpit Dashboard of the VPN module.

This information could be enhanced by extracting more data and persisting it; the goal is to aggregate these data over a period of time in order to analyze it and perform queries on it.

Some statistics of interest include:

Connection statistics should be accessible from these Cockpit modules:

Proposed solution

OpenVPN allows to execute a script during client connection and disconnection. VPN statistics can be persisted during these phases. Data can be saved on a lightweight database such as SQLite. The records on the database will be retrieved by:

Alternative solutions

Instead of using a relational database, VPN statistics could be persisted using a different strategy, such as:

nethbot commented 5 years ago

in 7.6.1810/testing:

nethbot commented 5 years ago

in 7.6.1810/testing:

nethbot commented 5 years ago

in 7.6.1810/testing:

nethbot commented 5 years ago

in 7.6.1810/testing:

andre8244 commented 5 years ago

Test case 1 (fresh install)

  1. Install nethserver-openvpn and nethserver-vpn-ui packages
  2. Configure a green and a red network interface (DHCP is ok for red)
  3. Access VPN application on Cockpit UI
  4. Access OpenVPN RoadWarrior page
  5. Click Enable OpenVPN RoadWarrior server
  6. Input Auth mode: Certificate, Mode: Routed, Network: a private network (such as 10.0.10.0), Netmask: the netmask of Network (such as 255.255.255.0), Contact this server on public IP / host: the green IP address configured. Click Save button
  7. Click Add account button and input Mode: VPN only, Username: a username of your choice. Click Save button.
  8. In RoadWarrior accounts table click the 3-dots (kebab) button of the account you have just created, click Download, then again Download button near OpenVPN configuration
  9. Open the downloaded OVPN configuration with a text editor and add route-nopull among the config lines at the top of the file. This step is needed in order to keep reaching your server
  10. Start a VPN connection (e.g. sudo openvpn path/to/ovpn/file/username.ovpn)
  11. Access VPN dashboard in Cockpit UI and verify that your VPN account name appears in Today top traffic accounts (OpenVPN roadwarrior statistics section)
  12. Access OpenVPN RoadWarrior page and verify that the start time of VPN connection appears in the Last connected column of RoadWarrior accounts table
  13. End VPN connection, refresh OpenVPN RoadWarrior page, click the link in Last connected column and check the information about your VPN connection
  14. Create at least another VPN account, then start and end a VPN connection
  15. Access VPN dashboard and verify that VPN accounts are sorted by traffic in Today top traffic accounts

Test case 2 (update)

  1. Update nethserver-openvpn and nethserver-vpn-ui packages
  2. Follow test case 1 from the second step

Test case 3 (weekly reports)

  1. Follow test case 1 steps to generate some VPN data
  2. Install nethserver-dante package
  3. Access Settings page of Report application on Cockpi UI
  4. Click Collect now button
  5. Go to Dashboard page, click Edit widgets button, then Default layout, then Edit done
  6. Verify that the widgets OpenVPN RoadWarrior account connections and OpenVPN RoadWarrior traffic appear and display correct data
dz00te commented 5 years ago

first quick test; Test Case 2, step 10-15, all OK for me

while i am on VPN dashboard in Cockpit UI, in log i see lot of

Sep 13 19:43:45 gasc cockpit-bridge: Argument "null" isn't numeric in int at /usr/libexec/nethserver/api/nethserver-vpn/dashboard/read line 100, <STDIN> line 1.
Sep 13 19:43:45 gasc cockpit-bridge: Argument "null\r" isn't numeric in int at /usr/libexec/nethserver/api/nethserver-vpn/dashboard/read line 100, <STDIN> line 1.
Sep 13 19:43:50 gasc cockpit-bridge: Argument "null" isn't numeric in int at /usr/libexec/nethserver/api/nethserver-vpn/dashboard/read line 100, <STDIN> line 1.
Sep 13 19:43:50 gasc cockpit-bridge: Argument "null\r" isn't numeric in int at /usr/libexec/nethserver/api/nethserver-vpn/dashboard/read line 100, <STDIN> line 1.
Sep 13 19:43:55 gasc cockpit-bridge: Argument "null" isn't numeric in int at /usr/libexec/nethserver/api/nethserver-vpn/dashboard/read line 100, <STDIN> line 1.
Sep 13 19:43:55 gasc cockpit-bridge: Argument "null\r" isn't numeric in int at /usr/libexec/nethserver/api/nethserver-vpn/dashboard/read line 100, <STDIN> line 1.
Sep 13 19:44:00 gasc cockpit-bridge: Argument "null" isn't numeric in int at /usr/libexec/nethserver/api/nethserver-vpn/dashboard/read line 100, <STDIN> line 1.
Sep 13 19:44:00 gasc cockpit-bridge: Argument "null\r" isn't numeric in int at /usr/libexec/nethserver/api/nethserver-vpn/dashboard/read line 100, <STDIN> line 1.
Sep 13 19:44:05 gasc cockpit-bridge: Argument "null" isn't numeric in int at /usr/libexec/nethserver/api/nethserver-vpn/dashboard/read line 100, <STDIN> line 1.
Sep 13 19:44:05 gasc cockpit-bridge: Argument "null\r" isn't numeric in int at /usr/libexec/nethserver/api/nethserver-vpn/dashboard/read line 100, <STDIN> line 1.

i will do some more tests... tnx

andre8244 commented 5 years ago

Hi dz00te,

It looks like your netdata API returns null values for sent and received data. Can you confirm that executing this command:

curl 'http://localhost:19999/api/v1/data?chart=net.tunrw&format=csv&before=0&after=-300&options=abs,seconds'

you get an output similar to this?

1568628418,null,null
1568628417,null,null
1568628416,null,null
1568628415,null,null

What versions of netdata and nethserver-netdata are you using?

gsanchietti commented 5 years ago

I could silence the warning, but IMO it's an indicator that netdata is not correctly collecting the data.

dz00te commented 5 years ago

just checked and i didn't have the errors in logs anymore... the only things i changed is the installation of ntehserver-dante. i've checket yum history no other packages was installed since other test. i'll try to replicate on a new installation, sorry. another strange things is in report today the widget OpenvpnRoadwarriorTraffic (chart) is empty. i'am sure it works yesterday... i'll do some other tests edit: sorry, vm was updated with latest nethserver-cockpit 0.14 i need to rollback...

dz00te commented 5 years ago

new test on another VM. Test Case 2, step 10-15, all OK, no dante installed I saw the same errors in the logs only for the first three times that I returned to the vpn menu, it seems that after a few minutes from the installation the error occurs in the logs no longer appears ...

you get an output similar to this?

 # curl 'http://localhost:19999/api/v1/data?chart=net.tunrw&format=csv&before=0&after=-300&options=abs,seconds'
time,received,sent
1568715044,0,0
1568715043,0,0
1568715042,0,0
1568715041,0,0
1568715040,4.106432,6.826944
1568715039,96.76146,111.64152
1568715038,33.46574,38.46223
1568715037,2.39437,2.757299
1568715036,0,0
.........

What versions of netdata and nethserver-netdata are you using?

# rpm -qa | grep netdata
netdata-data-1.16.0-1.el7.noarch
netdata-1.16.0-1.el7.x86_64
nethserver-netdata-1.1.0-1.ns7.noarch
netdata-conf-1.16.0-1.el7.noarch
cotosso commented 5 years ago

I tried the test cases, everything worked fine for me.

gsanchietti commented 5 years ago

it seems that after a few minutes from the installation the error occurs in the logs no longer appears ...

That is because netdata discover the data after a while.

nethbot commented 5 years ago

in 7.6.1810/updates:

nethbot commented 5 years ago

in 7.6.1810/updates:

nethbot commented 5 years ago

in 7.6.1810/updates: