search

Aharoni-Lab / Miniscope-DAQ-QT-Software

Software for streaming and controlling neural and behavioral data from freely behaving animals
GNU General Public License v3.0
119 stars 29 forks source link

Clarification on Timestamp Generation in Miniscope-DAQ-QT-Software #72

Open Lucas10pm opened 6 months ago

Lucas10pm commented 6 months ago

Hello,

I am using the Aharoni-Lab Miniscope-DAQ-QT-Software for live calcium imaging recordings and have a question regarding the timestamp mechanism used by the software. Specifically, I am trying to understand the starting reference point for the timestamps that are generated when recording videos.

The initial timestamp in some of our recordings, such as 935,703,178 milliseconds, does not seem to align with standard UNIX epoch time. I need to synchronize these timestamps with another system that logs events using a standard time notation (hh:mm:ss on specific dates).

Could anyone provide details or direct me to documentation that explains the origin or reference point from which the timestamps start counting?

Thank you!

sneakers-the-rat commented 6 months ago

Hello! Not the author of the software but let me see if I can help:

First question: which timestamps are you looking at? They are saved in a few places. Can you zip and attach an example json metadata/timestamp CSV file/whatever youre looking at?

When starting a recording, a QDateTime object is created with the current (system) time: https://github.com/Aharoni-Lab/Miniscope-DAQ-QT-Software/blob/0fecb97cca82c694b1b40dde03cd759e23ad3224/source/datasaver.cpp#L274

That gets stored in the recording metadata here: https://github.com/Aharoni-Lab/Miniscope-DAQ-QT-Software/blob/0fecb97cca82c694b1b40dde03cd759e23ad3224/source/datasaver.cpp#L477-L487

I believe the timestamps come from here: https://github.com/Aharoni-Lab/Miniscope-DAQ-QT-Software/blob/0fecb97cca82c694b1b40dde03cd759e23ad3224/source/datasaver.cpp#L214-L216

So the timestamps are taken from some timestamp buffer and then the time thats stored is the ms since the start time.

I believe the timestamp buffer times come from here: https://github.com/Aharoni-Lab/Miniscope-DAQ-QT-Software/blob/0fecb97cca82c694b1b40dde03cd759e23ad3224/source/videostreamocv.cpp#L201

So thats again just the current system time in ms since Unix epoch.

935,703,178ms is roughly 11 days - could it be the system time is off? Eg. A system without set time has been on for 10 days.

daharoni commented 6 months ago

Hey! So as @sneakers-the-rat said, the start time of the recording is saved in the metadata Json file. The actual timestamps in the timestamp file for each data source is saved as the number of milliseconds since the start of the recording. The first timestamp can be a little funky as there is a chance that that particular frame was acquired and sitting in the frame buffer of the software right before the start of recording. This causes the calculation of the millisecond value to be negative but our software improperly converts it to an unsigned int, resulting in a nonsensical large positive number.

This is a bug that we haven't gotten around to fix. Current options are:

  1. Just set the first timestamp to 0
  2. Just throwaway the first frame
  3. Recover the negative millisecond value by converting the unsigned int timestamp back to a signed one

I hope this helps.