Data was not rendering correctly due to low precision rounding in dat…

[tstrutz]

I've been using your library and have been happy with it for the exception of this case. For example with an X vector: [0 , 1e-9... ... 1e-6 - 1e-9, 1e-6] with 1ns steps, all of these coordinates will round to zero instead of keeping their true precise value, due to a hidden rounding feature in the library. This causes my plots to look something like this:

To fix, I increased precision of all data strings to 10 digits. Was using data sampled at about 0.8ns and it was putting the first 10000+ points at the zero X value instead of where they should have been placed. This commit fixes the issue (at probably a small performance impact). I imagine a better fix for this would to be making the setprecision usertunable. Is this hard to do with the current implementation?

This is what it looks like with the correct implementation.

I am using sampled data at over 1 billion samples per second (requires > 1ns x-axis accuracy), so this precision is potentially needed unless I convert the units, but it broke due to a feature I didn't even know was in the library!

Thanks,

Tim

[alandefreitas]

At this point, setting a higher precision seems like a good solution. This is a limitation in the interaction with Gnuplot, but from the POV of the API, the user has already set the precision by providing the data.

[alandefreitas]

Thanks ! :)