opencadc / caom2tools

Common Archive Observation Model - data engineering tools
2 stars 13 forks source link

ObservationWriter lossy writing of floats #47

Closed dr-rodriguez closed 7 years ago

dr-rodriguez commented 7 years ago

When CAOM observations are written to XML files, floats may be truncated in precision. This results in checksum inconsistencies when comparing stored checksums to those computed after reading the XML. As an example, a float value of -0.00518884856598203 will get stored as -0.00518884856598 A temporary measure would be to calculate checksums after the file is written, but ideally we would want to preserve the full precision of any quantities provided.