pacificclimate / Rudunits2

GNU General Public License v2.0
26 stars 10 forks source link

Numerical inaccuracies #29

Open bersbersbers opened 3 years ago

bersbersbers commented 3 years ago

The following outputs -1.11022302462516e-16 instead of 0:

udunits2::ud.convert(1, "in", "in") - 1

This leads to cairo_pdf saving images with size 71 pt x 71 pt rather than 72 pt x 72 pt according to pdfinfo, which is quite visible in exactly aligned images.

I wonder if trivial conversions (in -> in, for example) could be skipped to maintain accuracy.