Ecephys extract running speed broken for Visual Behavior Neuropixel data

[wbwakeman]

Issues encountered when doing 'extract running speed' for ecephys. Need to investigate these issues and create tickets to address them.

Issue #1: sync file line label has been updated from stim_vsync to vsync_stim as described in the excel file link in "Sync: PCIe 6612 DAQ" section of this confluence page .

https://github.com/AllenInstitute/AllenSDK/blob/master/allensdk/brain_observatory/sync_dataset.py#L96

Traceback (most recent call last):
  File "/allen/aibs/technology/conda/production/allensdk_py36/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/allen/aibs/technology/conda/production/allensdk_py36/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/allen/aibs/technology/conda/production/allensdk_py36/lib/python3.6/site-packages/allensdk/brain_observatory/extract_running_speed/__main__.py", line 146, in <module>
    output = main(**mod.args)
  File "/allen/aibs/technology/conda/production/allensdk_py36/lib/python3.6/site-packages/allensdk/brain_observatory/extract_running_speed/__main__.py", line 100, in main
    "rising", Dataset.FRAME_KEYS, units="seconds"
  File "/allen/aibs/technology/conda/production/allensdk_py36/lib/python3.6/site-packages/allensdk/brain_observatory/sync_dataset.py", line 384, in get_edges
    f"none of {keys} were found in this dataset's line labels")
KeyError: "none of ('frames', 'stim_vsync') were found in this dataset's line labels"

---

Issue #2: sync mismatch

https://github.com/AllenInstitute/AllenSDK/blob/master/allensdk/brain_observatory/extract_running_speed/__main__.py#L112

Need to make sure a fix here does not break any two-photon running speed processing

Traceback (most recent call last):
  File "/allen/aibs/technology/conda/production/allensdk_py36/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/allen/aibs/technology/conda/production/allensdk_py36/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/allen/aibs/technology/conda/production/allensdk_py36/lib/python3.6/site-packages/allensdk/brain_observatory/extract_running_speed/__main__.py", line 146, in <module>
    output = main(**mod.args)
  File "/allen/aibs/technology/conda/production/allensdk_py36/lib/python3.6/site-packages/allensdk/brain_observatory/extract_running_speed/__main__.py", line 112, in main
    f"found {num_raw_timestamps} rising edges on the vsync line, "
ValueError: found 215999 rising edges on the vsync line, but only 91201 rotation samples

Investigation of this issue can be timeboxed to 3 days. May include running another experiment through this to ensure it happens with other experiments.

[corbennett]

I think it's likely that this is due to the fact that we have 3 pkl files for this experiment (behavior, mapping and replay; all with running data). My guess is that this code is only finding running data for the 'stim.pkl' which is the mapping portion of our experiment. This is 25 minutes long, which matches the number of running samples that are found. We will need to take the running data from all three pkls (or concatenate the pkl files before running them through this code).

[wbwakeman]

Useful information for Wayne:

lims2=> SELECT wkft.name AS wkft, wkf.filename AS wkf FROM well_known_files wkf JOIN well_known_file_types wkft ON wkft.id=wkf.well_known_file_type_id WHERE wkf.attachable_id = 1125713722 AND well_known_file_type_id IN (1007316668,1105900356,610487715);
         wkft          |                   wkf
-----------------------+-----------------------------------------
 StimulusPickle        | 1125713722_578257_20210901.behavior.pkl
 MappingPickle         | 1125713722_578257_20210901.mapping.pkl
 EcephysReplayStimulus | 1125713722_578257_20210901.replay.pkl

Example of current ECEPHYS_EXTRACT_RUNNING_SPEED input.json file:

{
"sync_h5_path": "/allen/programs/braintv/production/visualbehavior/prod0/specimen_1109521187/ecephys_session_1132595046/1132687142/1132595046_583430_20211006.sync",
"stimulus_pkl_path": "/allen/programs/braintv/production/visualbehavior/prod0/specimen_1109521187/ecephys_session_1132595046/1132687142/1132595046_583430_20211006.mapping.pkl",
"output_path": "/allen/programs/braintv/production/visualbehavior/prod0/specimen_1109521187/ecephys_session_1132595046/EcephysExtractRunningSpeedStrategy/analysis_run_1149574448/1132595046_running_speeds.h5"
}