Bad times in rtplot when plotting ULTRASPEC drift mode frames

[vikdhillon]

When rtplotting ULTRASPEC data taken in drift mode with first=0, one gets junk times, which all have the value 2000-01-01 00:00:00.000. When using first=1, the times are fine. When using first>1, rtplot crashes with the error shown in the attached photo.

[trmrsh]

I have merged a load of fixes from trm-dev into the master branch as I think they are useful. This problem should be fixed. Partly a problem caused by hipercam flagging many valid times as "not OK". Reduce files now have an option 'skipbadt' to skip bad times which should be set = yes for ultracam and ultraspec. Also added a script 'rupdate' to enable updating of old reduce files. I am leaving the issue open to remind me to look in more detail at the drift mode times.

[vikdhillon]

Hi Tom,

Just tested the new pipeline with a drift mode. The times still seem to be 2001-01-01 etc in rtplot. We tried and failed to find a corresponding skipbadt option in rtplot - does that need to be added?

Cheers,

Vik.

---

Prof. Vik Dhillon, Dept of Physics & Astronomy, Univ of Sheffield, Sheffield S3 7RH, UK +44 114 222 4528; www.vikdhillon.staff.shef.ac.uk

On Mon, 24 Feb 2020 at 19:54, Tom notifications@github.com wrote:

I have merged a load of fixes from trm-dev into the master branch as I think they are useful. This problem should be fixed. Partly a problem caused by hipercam flagging many valid times as "not OK". Reduce files now have an option 'skipbadt' to skip bad times which should be set = yes for ultracam and ultraspec. Also added a script 'rupdate' to enable updating of old reduce files. I am leaving the issue open to remind me to look in more detail at the drift mode times.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/HiPERCAM/hipercam/issues/78?email_source=notifications&email_token=AC46DHGY5YABULRGGSYFPITREQQZBA5CNFSM4K2LSQRKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEMZJVPI#issuecomment-590518973, or unsubscribe https://github.com/notifications/unsubscribe-auth/AC46DHF3CXAND4WIF44KQLDREQQZBANCNFSM4K2LSQRA .

[vikdhillon]

Ah - just noticed that the bad times are only present if you do first=0. If you do first=500 or whatever, it works fine. Is that expected behaviour due to the fact that only a fraction of the frames are displayed in rtplot in drift mode when first=0, so no lookback times are available in the time calculation?

---

Prof. Vik Dhillon, Dept of Physics & Astronomy, Univ of Sheffield, Sheffield S3 7RH, UK +44 114 222 4528; www.vikdhillon.staff.shef.ac.uk

On Tue, 25 Feb 2020 at 11:00, Vik Dhillon vik.dhillon@sheffield.ac.uk wrote:

Hi Tom,

Just tested the new pipeline with a drift mode. The times still seem to be 2001-01-01 etc in rtplot. We tried and failed to find a corresponding skipbadt option in rtplot - does that need to be added?

Cheers,

Vik.

---

Prof. Vik Dhillon, Dept of Physics & Astronomy, Univ of Sheffield, Sheffield S3 7RH, UK +44 114 222 4528; www.vikdhillon.staff.shef.ac.uk

On Mon, 24 Feb 2020 at 19:54, Tom notifications@github.com wrote:

I have merged a load of fixes from trm-dev into the master branch as I think they are useful. This problem should be fixed. Partly a problem caused by hipercam flagging many valid times as "not OK". Reduce files now have an option 'skipbadt' to skip bad times which should be set = yes for ultracam and ultraspec. Also added a script 'rupdate' to enable updating of old reduce files. I am leaving the issue open to remind me to look in more detail at the drift mode times.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/HiPERCAM/hipercam/issues/78?email_source=notifications&email_token=AC46DHGY5YABULRGGSYFPITREQQZBA5CNFSM4K2LSQRKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEMZJVPI#issuecomment-590518973, or unsubscribe https://github.com/notifications/unsubscribe-auth/AC46DHF3CXAND4WIF44KQLDREQQZBANCNFSM4K2LSQRA .

[trmrsh]

yes, with first=0, the bad times in rtplot are expected. It can't generate correct times without the preceding timestamps. The important thing to test is 'reduce'.

[trmrsh]

Does 'reduce' work now with drift mode data?

[trmrsh]

I am closing this on the basis that I think (a) reduce works which was the main issue, and the times are genuinely bad for the first frames of drift mode, (b) one still want rtplot to plot frames even if they are bad. I might add an indicator of whether the time is flagged as bad which does not currently seem to be the case in rtplot.