search

inpta / pinta

Data analysis pipeline for upgraded GMRT pulsar data
GNU General Public License v2.0
3 stars 0 forks source link

MJD precision issue #5

Closed abhisrkckl closed 3 years ago

abhisrkckl commented 4 years ago

Precision in MJD is too large. We need up to 200 ns only (2e-12) so only 13 decimal places are needed. Why MJD as 58260.122689274459844455 ? Why not 58260.1226892744598? That also results in slightly off timestamps. e.g 58260.122689274459844455 instead of 58260.12268927445926

abhisrkckl commented 4 years ago

We need only 14 decimal point precision (~1 ns) in MJD. Also, the precision of the MJD computed using astropy should be tested.

abhisrkckl commented 4 years ago

BCJ: Timestamp Precision is OK now. I tested filterbank header against manual - it is OK for some files and wrong for some files.

abhisrkckl commented 4 years ago

BCJ: I processed a data set that starts at 3 am IST and the timestamp was correctly adjusted to MJD minus one day

abhisrkckl commented 4 years ago

Issue summary ============

  1. The original issue identified by BCJ was that the timestamp in the filterbank header (accessed through the 'header' command) is different from the hand-computed value from the GMRT timestamp file. The error is >100 ns.

  2. I identified this to be a floating-point precision issue in python. Python 'float' type can hold only 15 significant digits which are not enough to store the timestamp MJD. I fixed this by separately computing the integer and decimal parts of the timestamp and sticking them together in a string so that floating-point accuracy is not a limiting factor.

  3. BCJ identified a bug in the new string-based code which caused large errors in some timestamps. This was also fixed. (This version is now being tested.)

  4. The MJD in the filterbank header was still not coming accurately. I verified that the correct timestamp was being passed into the filterbank command. Furthermore, the filterbank files produced by rfiClean also had the same issue.

  5. Yogesh explained that the timestamp is stored as a double in the filterbank file which doesn't provide enough accuracy for our purposes. He suggested passing the timestamp directly to dspsr using -m option.

  6. I implemented the -m option in pinta, which is available as a "beta" version. However, we are still unable to verify the correctness of the timestamps in the FITS file as the relation between the start MJD and int:mjd values in the FITS file is unclear. This is the current situation.