Smoothed lines option causing some curves between points to be unrealistic

[joshuar]

Sorry, hard to describe other than show a visual:

Looks like the function used to calculate the curvature of the line between two points has an issue here for the repeater.fritz.box line. Raw data export here. The line in question is for target_host=repeater.fritz.box

[joshuar]

Refreshed the display and got more funkiness:

Uploaded raw data here

[stormpython]

@joshuar This has to do with the fact that we are using the Cardinal Spline algorithm for smoothing lines. It was chosen because it had a nicer smoothing effect than some of the other algorithms. But in light of this, perhaps we should change it.

That being said, anytime you smooth lines when it comes to displaying data, you are essentially lying about the data, as the most accurate representation is a linear interpolation between points.

[joshuar]

@stormpython yeah, the fit seems to be worse when there are fewer data points and they are not evenly spaced. For that data, its ICMP ping round-trip times and that graph is only showing the hosts with RTT above 50ms. For the bad fitting graphs, those hosts have ping times that jump above and below 50ms, hence the uneven spacing and few data points.

Yeah, line fitting like this can be a delicate art :) Not sure if the answer is to allow a user to select the different fits themselves or just put a warning when the option is enabled that the line fitting can lead to inaccurate results?

[sqpdln]

this is addressed here as well: https://github.com/elastic/kibana/issues/3737, i've written a comment with another example.

[JulienPalard]

Another funky one:

[stormpython]

Upon further investigation, none of the out-of-the-box d3 interpolation functions solve this particular problem. For more information on the d3 interpolation functions see: d3 interpolators.

There are two solutions as I see it currently:

1. Use linear interpolation
2. Create our own linear interpolation function (or steal someone else's that solves this problem)

I am removing the discuss tag, and perhaps another issue can be opened as an enhancement for line charts.

[arabold]

You run into the problem of "funky curves" quite easily if your data points aren't coming in regular intervals. Here's an example where I'm just missing data and thus the curves go crazy:

A valid possibility would be to show gaps where data points are missing. Another good food for thoughts is here: http://www.d3noob.org/2013/01/smoothing-out-lines-in-d3js.html. Maybe it would be best we offer multiple choices (drop-down) instead of a simple checkbox to smoothen lines.

[pemontto]

I have another example here where I have cluster of data and then the next point is far in the future, it's like I've pulled a bezier curve control point too far.

[ppisljar]

looking at the https://github.com/d3/d3-shape/blob/master/README.md#curves monotone seems to be better choice than cardinal

[ppisljar]

i tested the monotone interpolation and it also produces weird lines

[ppisljar]

all the other interpolation types that d3v3 supports do not draw the line thru all the actual data points, which will produce weird results if you are also plotting the circles