Define how to bundle/interpolate data from different sensors

[ezio-melotti]

Different sensors read data at different times and with different intervals. It would be convenient for the frontend to receive all values at once, with the same step number and timestamp. In order to do that we need to bundle or interpolate the data. I can see three solutions:

1. cache the last value for each type, and send that (easier)
2. if we send data with a small delay, we can interpolate the last two (or more) readings;
3. if we don't, we will have to extrapolate the value

To give you an example, assume we have a reading every 1.5s, and we are sending data to the frontend every 1s. At 10s, the frontend has to send a value, but the last reading was at 9s (0s, 1.5s, 3s, 4.5s, 6s, 7.5s, 9s) and the next will be at 10.5s.

1. just send the last reading (taken at 9s, i.e. 1s ago)
2. wait a couple of seconds and then interpolate the 9s and the 10.5s readings, and send the resulting 10s value with a small delay
3. extrapolate the 10s value based on the previous readings (i.e. 9s, 7.5s, ...)

The final object emitted by the socketio server could look like:

{"n": ...,
 "timestamp": ...,
 "readomgs": {
   "scd30": {"co2": ..., "temp": ..., "humidity": ...},
   "bme668": {"pressure": ..., "temp": ...},
   ...
 }
}

The list of available sensors should be sent by the server when the frontend connects. We might want to add an id or label to the sensor name in order to distinguish multiple sensors of the same type (e.g. SCD30-0 or SCD30-greenhouse).

[ezio-melotti]

Currently the sensor sends batches to the server, and the server immediately broadcasts them to the clients. This means that all the clients get the same batch (roughly) at the same time. With multiple sensors we can't immediately broadcast, but we need to store the data somewhere when we receive them from the sensor and then combine them in a bundle before sending them to the client.

Now, there are two options:

1. The server has a single loop that creates at fixed time intervals a bundle (or a batch of bundles) with stored data from all sensors and broadcasts it to all clients at (roughly) the same time. All the clients will receive the same bundles/batches with the same step number and timestamp at the same time, and they won't be able to request data with higher or lower frequency.
2. The server has a separate loop for each client, each client can specify a different delay and can get "custom" bundles/batches with their own step number and timestamp. This is more flexible, but more computationally and memory intensive, since each loop will have to create its own bundles with a custom step number with a custom frequency. If the bundles are batched, each loop will also have to store in memory the last N bundles, if not, a bundle with the last readings can be created and sent.

An hybrid approach could also be possible: the server could creates bundles every second (or maybe half second) and store them in a global batch. Each client loop can then create a custom batch by extracting bundles from the global batch. For example, if the global batch has the last 100 bundles with a 1s interval, a client with 1s delay and a batch size of 10 bundles could get the last 10 bundles from the global batch, put them together in a custom batch with a custom step number, and repeat the process every 10s. A client with a 5s delay and a batch size of 3 bundles could get the bundles -1, -6, -11, and repeat the process every 15s (3 * 5s). This provides some flexibility while avoiding being memory-intensive, even though the frequency and batch size must be compatible with the global bundle.