Forcing data types in InfluxDB

[mariofrei]

Inserting data into an existing InfluxDB column requires the new data to be of the same type as the existing data within a shard. This can be problematic if the data type is not enforced as it might change. Example:

• Inserting data from a backup CSV file where integers might be read as floats
• Inserting data from HealthKit that can switch between float and integer, e.g., when zero.

Currently, the datatype is enforced in the lambda function cozie-apple-app-write. This issue is also one of the reasons why we decided to have a new 'measurement' for each 'experiment ID'

To make things simpler and more consistent, we could consider forcing the datatype float for all numerical values. That would help avoid some horrific surprises when inserting new data.

[mariofrei]

Alternatively, the data type could be enforced automatically by modifying the payload format as shown below. This way, no changes to the backend need to be made if and when new fields are implemented in the app.

Current format:

[
   {
      "time":"2022-09-01T07:02:51.578+0800",
      "measurement":"dev",
      "tags":{
         "id_onesignal":"35E2A783-35DA-4C5F-B54E-5DAC30B6E860",
         "id_participant":"dev01",
         "id_password":"5DAC30B6E86"
      },
      "fields":{
         "ts_heart_rate": 89,
         "ts_oxygen_saturation": 99,
         "ts_walking_distance": 203.89,
         "ts_workout_type": "Functional Strength Training"
      }
   }
]

New format:

[
   {
      "time":"2022-09-01T07:02:51.578+0800",
      "measurement":"dev",
      "tags":{
         "id_onesignal":"35E2A783-35DA-4C5F-B54E-5DAC30B6E860",
         "id_participant":"dev01",
         "id_password":"5DAC30B6E86"
      },
      "fields_integer":{
         "ts_heart_rate": 89
         "ts_oxygen_saturation": 99,
      },
      "fields_float":{
         "ts_walking_distance": 203.89,
      },
      "fields_string":{
         "ts_workout_type": "Functional Strength Training"
      }
   }
]