tguild
commented
5 months ago Hej @eriksven unless there is separate logic to perform an in-vehicle assessment, the curve sampled data point is considered relevant and should be transmitted back to the cloud. It is a lossy compression algorithm that sends pertinent data based on the error thresholds per signal. The new data point may be actionable and should be sent to the cloud in the relatively near future but can be kept in a buffer and sent as part of a batch. Note some of our thresholds could potentially be better tuned, we have some default thresholds for temperature and pressure that can studied and improved.
@yuhanlin-geotab would know better and may want to expound further or correct impressions I have.
yuhanlin-geotab
eriksven
For the guidelines, I am wondering when which data is actually send from the vehicle to a backend and how much delay we allow between the measurement and the transfer of the sample to a backend. What do you think?
For the curve algorithm, which is used for many data points like the GPS location, I see different options: "slice" the data either with or without overlap and curve or curve after each new sample.
For reference, let us assume the following samples:
When we apply the curve algorithm over all data we could get the following result.
However, we are dealing with a stream of data and do not have an infinite buffer size to fit all potential samples and also need to make a decision when to send the data to the backend (before infinity). One approach would be to slice the data set and then apply curve on each slice. We can then send the result per slice directly to the cloud. This would look like this:
In this approach, we treat each buffer independently and will always send at least the first and last sample of each slice. A specialization of this would be to let the slices overlap each other so that the last point of one slice is the first point of the next slice. This would like this:
Another approach is to apply curve whenever a new sample gets added to the buffer. Here we could decide for an each sample whether we keep it and therefore send it to the cloud after curve has been applied for the subsequent sample.
To avoid that the buffer size becomes too large, we could work with a sliding window and drop the oldest samples from the buffer after pushing it to the cloud.
A subsequent question is whether to send the sample always in the moment it is decided that we keep it or to collect a number of samples before sending them. So one could only send the relevent samples every minute etc. or when a local storage is full.