LFP error for time stamps

[jeromelecoq]

Some LFP files on DANDI for Glow have 24s period instead of about 0.08 s

[jeromelecoq]

@Ahad-Allen, we were applying newer code for LFP timestamps to older files. This is related to OpenEphys changing version. @Ahad-Allen link code location where the error happens.

[Ahad-Allen]

We regenerated the files for mouse 647836 https://dandiarchive.org/dandiset/000253/draft/files?location=sub-647836&page=2. @rcpeene was attempting to review them, but dandi is down sadly.

[jeromelecoq]

Now that dandi is good. Carter can check.

[Ahad-Allen]

We were looking at the input lfp timestamps for these sessions and found some strange behavior. There are a range of unexpected deltas, including negative values

These values do not seem to track with the 1250hz rate that is expected, either.

When carter plotted these post subsampling, he found that it seems that each negative value has a positive mirror:

Jerome mentioned that this probably a result of the change to open ephys in the middle of these projects, and the timestamps may need to be adjusted using the sample numbers.npy approach we had to use for some other sessions. These currently don't have these assets in their uploads, so I will reach out to Ben to see what's going on here.

[Ahad-Allen]

After some further investigation, I see that these sessions do have their sample numbers.npy present! They just also have their upload for lfp_timestamps.npy. I am going to reach out to Ben to confirm, but we should be able to use the files present here in that case: /allen/programs/mindscope/production/openscope/prod0/specimen_1210326253/ecephys_session_1221092548/1221092548_642507_20221026_probeABC/Record Node 101/experiment1/recording1/continuous/Neuropix-PXI-100.probeA-LFP

[jeromelecoq]

Ben has confirmed this is the problem. @Ahad-Allen needs to change the code to reflect this change of array.

[jeromelecoq]

@Ahad-Allen to change files to point arrays to samples.

[jeromelecoq]

When you point array to use sample number in ecephys alignment modules. Output timestamps still have negative number.

@Ahad-Allen to check if number are not negative in the raw data.

[Ahad-Allen]

Essentially, we need to

1. Grab the sample number for lfp start from sample_numbers.npy
2. Gen a list of LFP samples assuming the fps
3. Take those samples and use them as the indices we point to in ecephys_align_timestamps
4. Take that output and use it as the LFP times in the NWB

it seems that for these GLO files, we did not correct point toward these output times and instead packaged the data with the samples we generated as the timestamps, which resulted in the timestamps just being the samples. The negative values were the result of pointing the wrong LFP samples as the INPUT for ecephys_align_timestamps when debugging, this was an error on my part. I've plotted and found no negative values or deltas now. Carter is looking at sub-647836/sub-647836_ses-1227858756test_probe-0_ecephys.nwb to test

[rcpeene]

Timestamps look good for this file

[jeromelecoq]

@Ahad-Allen to add feedback from Eli and some of latest posts from him

[Ahad-Allen]

Eli noticed a new possible misalignment of times when comparing the new lfp file to the old ecephys one. We've asked him to compare the new and old ecephys nwb files to see if the issue is due to a change in the file somehow: