search

catalystneuro / buffalo-lab-to-nwb

Scripts to convert Buffalo Lab data to the NWB standard
2 stars 0 forks source link

all times should be in seconds after session_start_time #15

Closed bendichter closed 4 years ago

bendichter commented 4 years ago

the spike times are not synced. Luiz, can you please fix this?

bendichter commented 4 years ago

@luiztauffer When I run it on 2017_5_04 the times for EyeTracking and Position start at 2076786. Can you please make fixing this your highest priority? I'd like to finish this project.

luiztauffer commented 4 years ago

You're right, it is off like that. The "reference time" is still being discounted, however, it's just that it's too small for this 2017_5_04 file. t0 is being taken from the nex5 file:

file_data = nexfile.Reader(useNumpy=True).ReadNexFile(nex_file_name)
t0 = file_data["FileHeader"]["Beg"]

this field is described in nex5 documentation as "file data start in ticks", so I thought it would be reasonable to use it as reference, it is also compatible with the first timestamps for all data in the 2017-04-27 dataset. But maybe this info is not reliable and we should retrieve it from somewhere else. The first spike times for units taken from 2017_5_04 nex5 are also around 2075304, which is good because it seems to be aligned with with behavior timestamps.

if there's not a single reliable source of experiment start time in these files, I'm not sure how to proceed. Any ideas?

bendichter commented 4 years ago

Thanks for the explanation. This is why we asked for a second session! It sounds like your original approach was reasonable, but doesn't always generalize. So we have eyes, position, LFP, and spikes all on the same clock, right? The problem is that the start value is arbitrary. I think we can say that the session_start_time is the time of the first measurement across all of these modalities. i.e. (pseudo-code)

t0 = min(eyes.timestamps[0], pos.timestamps[0], lfp.timestamps[0], min(spike_times))
...
dat.timestamps = dat.timestamps - t0
luiztauffer commented 4 years ago

The latest changes should resolve this now. One thing to notice is that Position starts a bit later, this is because there’s no recording of it for calibration trials: https://github.com/ben-dichter-consulting/buffalo-lab-data-to-nwb/blob/a5f1520e078df0d5cd4bfa3f8187c678f46c20a3/src/buffalonwb/add_behavior.py#L32

EyeTracking, units spikes and voltage traces however seem to be recorded during calibration trials already, by looking at their start times and duration.

bendichter commented 4 years ago

Yes, this makes sense. The calibration is showing images and it is for calibrating the eye tracking instrument, we should not expect position data during those epochs. Thanks for taking care of this!

On Sun, Dec 8, 2019 at 1:02 PM Luiz Tauffer notifications@github.com wrote:

The latest changes should resolve this now. One thing to notice is that Position starts a bit later, this is because there’s no recording of it for calibration trials:

https://github.com/ben-dichter-consulting/buffalo-lab-data-to-nwb/blob/a5f1520e078df0d5cd4bfa3f8187c678f46c20a3/src/buffalonwb/add_behavior.py#L32

EyeTracking, units spikes and voltage traces however seem to be recorded during calibration trials already, by looking at their start times and duration.

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub https://github.com/ben-dichter-consulting/buffalo-lab-data-to-nwb/issues/15?email_source=notifications&email_token=AAGOEET2F7JT22GXB7D5TKLQXVOHBA5CNFSM4JE6I2SKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEGHJRRI#issuecomment-562993349, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAGOEEU6VGHZ4O2HDQD3V3TQXVOHBANCNFSM4JE6I2SA .

-- Ben Dichter, PhD Data Science Consultant