PINT accuracy goal?

[paulray]

I think PINT should have an explicitly-stated accuracy goal of some number of nanoseconds.

Then, the test suite should verify that various components of the calculations meet that goal, wherever it is possible to construct a test with a known precise answer.

There should also be validation against TEMPO or Tempo2 results. It is not always clear what inconsistencies mean (which code is correct?), but they should be documented somehow. These tests might mostly be done outside of the travis-ci testing, but I know that some are already there, based on pre-computed files of TEMPO/Tempo2 results. This is a good start, but the point of the accuracy goal is that is sets the scale for what is 'close enough'. As I've been debugging my own photon phase calculations, it is clear that validations get progressively harder as you go from 1 µs to 100, 10, or 1 ns. It is important to know what the accuracy goal is to know how hard to work on any particular aspect of the calculation.

[paulray]

At the telecon today, we decided that 1 ns was really hard and 10 ns is more realistic, but also that one number can't really characterize the accuracy goal since there are many different types of errors. The difference between different JPL ephemerides is on the 10 ns or larger scale, for example.

The PINT paper should definitely have a section on Error Budget that reviews the accuracy in various parts of the calculation (time transformations, ephemerides, delay calculations, floating point accuracy, etc...)

[paulray]

@luojing1211 is this addressed sufficiently in the paper that we can close this issue? I do think it is important to have some statement of the accuracy goals in there.

[aarchiba]

There's still the other half of the problem - adding test cases (whether they work in Travis or not) that confirm that we meet the accuracy goal. How well are those implemented?

[paulray]

Yeah, this would be a great contribution from a NANOGrav student