Grouping of subsamplig to provide minimum maximum values

[santilland]

As commented i think this issue needs some discussion i will write some thoughts here to try elaborating what the requirements are.

[santilland]

So after some discussion with colleagues i think there are some points that need to be considered. I will try to describe the different first thoughts in parallel to an examples.

Let's say we select 1 day of 1Hz data, so we have 86400 points and apply a 20s subsampling resulting in 4320 points. We do this by just reading every 20th point contained in the data. The requirement is now to make sure no spikes are in the remaining 19s sections between the selected points. What should the approach be to fulfill this requirement?

• Approach 1 The "easiest" option would be to extend the grouping size to double the sampling e.g. 40s and extract minimum and maximum and return 2 measurements. Some of the problems here would be:

  • do we keep the timestamp of the original subsampling steps? (so every 20 seconds) or do we use the timestamps of the min/max?
  • What is the approach considering products have multiple parameters? The min/max for each parameter can be at different measurements inside the 40s group
  • how are multiproduct selections handled? Maybe taking the timestamps from the "master" product?

• Approach 2 We take the min/max inside the sampling group doubling the resulting, this would only work if we set a master parameter which defines what timestamp is used to get all other parameters (as the min/max positions will probably not coincide most of the time). Then we could maybe define the master parameter for each collection/type? For multiple products we could then use the timestamp of the master parameter of the master product. After some thought this seems to be maybe the most reasonable one, although i'm not sure if there should be a possibility for the user to define the master parameter?

• Approach 3 We actually introduce a new "interval" concept, meaning we no longer have single points/measurements (or we have both things depending on selected time) but start/end intervals which we could visualize and handle differently in the client, which would mean additional to the changes on the server to also change the client. This would need to be discussed further anyway so i won't go into more detail here.

For any approach in any case we need to consider that we will need to access every measurement and apply some function to it, which was not necessary for subsampled visualization until now, this probably means a noticeable difference in performance, especially when looking into visualizing longer time periods.

Ok, so there are my first thoughts, we can discuss more here or through other channels.

[lmar76]

After internal discussion we decided to discard this new requirement because the user must be aware that a subsetting process is applied on the visualized data.

Maybe could it be useful to show this information to the user, e.g.: number of points currently visualized, subsetting step, etc.

[santilland]

I think we should be able to return to the client the current subsampling step size and show it somehow on the client, we will discuss some options and propose a solution

[lmar76]

I agree. Maybe a status bar at the bottom of the page showing the number of points and the step...and additional information if needed.