Review of data types required for all test results as many use wrong data type

[wifinigel]

An issue has come to light with the data types being used for results data in InfluxDB. Many data fields are incorrectly using string types, when they should be integer of float types. This is due to incorrect coding in the results exporter .

Here is an example of an incorrect table from Influx:

> show tag keys
name: wiperf-ping
tagKey
------
Host

> show field keys
name: wiperf-ping
fieldKey     fieldType
--------     ---------
percent_loss string
ping_host    string
ping_index   integer
pkts_rx      string
pkts_tx      string
rtt_avg_ms   float
rtt_max_ms   float
rtt_mdev_ms  float
rtt_min_ms   float
test_time_ms string

This is a high priority issue as it prevents reporting calculations from being performed in customised reports. Fixing this will also break previous database structures & wiperf instances that will continue to use their existing DB structures. This will have to be a v2.1.x release, as all existing v2.0.x releases will be broken.

All data exports needs reviewing & fixing in wiperf code before further features and releases are made available.

[wifinigel]

....suspect this may also affect Splunk exports, but not tested as yet.

[wifinigel]

All modules which export data will need to gave their data types reviewed. This will have the side effect of meaning that existing time-series data will need to be dropped so that exports will continue to work with the corrected data field types. This has a undesirable impact for users with existing data as they will have to re-initialise their DB tables, which will result in loss of historic data.

[wifinigel]

All modules reviewed and updated as appropriate so that all data points that are exported have a definite data type (i.e. string, integer, float).

Review showed that only the ping test results were causing an issue , all others were correct type in InfluxDB.

When data types are fixed in v2.1, found that errors are caused when wiperf tries to export ping test results - does not cause a catastrophic failure, just a warning that data export to InfluxDB has failed for just that test. This issue does not cause any export issues to Splunk when v2.1 code is installed and there is existing ping test data in the Splunk DB.

The fix for InfluxDB that has existing ping data is to drop all existing ping test data:

This fixed issue will cause failures for ping test results on existing data. To correct the issue, existing ping 
test data in the Influx DB must be dropped using the following command in the InfluxDB CLI utility:

   sudo influx -username admin -password letmein
   use wiperf
   drop series from "wiperf-ping"