the number of stimulus frame times in the sync file does not agree with the number of photodiode events

[farznaj]

Describe the bug For a mesoscope session, dataset.ophys_timestamps fails.

To Reproduce

import visual_behavior.data_access.loading as loading
#experiments_table = loading.get_filtered_ophys_experiment_table() 
experiment_id = 945586450   # session_id = 945124131
dataset = loading.get_ophys_dataset(experiment_id, include_invalid_rois=False)
dataset.ophys_timestamps

Expected behavior To get an output from dataset.ophys_timestamps

Actual Behavior I get the following error:

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-98-a8ceda67fad6> in <module>
----> 1 dataset.ophys_timestamps
c:\users\farzaneh.najafi\documents\analysis_codes\visual_behavior_analysis\visual_behavior\data_access\loading.py in ophys_timestamps(self)
    359     @property
    360     def ophys_timestamps(self):
--> 361         if super().metadata['rig_name'] == 'MESO.1':
    362             self._ophys_timestamps = self.timestamps['ophys_frames']['timestamps']
    363         else:
c:\users\farzaneh.najafi\documents\analysis_codes\allensdk\allensdk\brain_observatory\behavior\behavior_ophys_session.py in metadata(self)
    137         """
    138         if self._metadata is None:
--> 139             self._metadata = self.api.get_metadata()
    140         return self._metadata
    141 
c:\users\farzaneh.najafi\documents\analysis_codes\allensdk\allensdk\api\cache.py in wrapper(*args, **kwargs)
     73         if result is not sentinel:
     74             return result
---> 75         result = f(*args, **kwargs)
     76         cache[key] = result
     77         return result
c:\users\farzaneh.najafi\documents\analysis_codes\allensdk\allensdk\internal\api\behavior_ophys_api.py in get_metadata(self)
    109         metadata['experiment_container_id'] = self.get_experiment_container_id()
    110         metadata['ophys_frame_rate'] = self.get_ophys_frame_rate()
--> 111         metadata['stimulus_frame_rate'] = self.get_stimulus_frame_rate()
    112         metadata['targeted_structure'] = self.get_targeted_structure()
    113         metadata['imaging_depth'] = self.get_imaging_depth()
c:\users\farzaneh.najafi\documents\analysis_codes\allensdk\allensdk\api\cache.py in wrapper(*args, **kwargs)
     73         if result is not sentinel:
     74             return result
---> 75         result = f(*args, **kwargs)
     76         cache[key] = result
     77         return result
c:\users\farzaneh.najafi\documents\analysis_codes\allensdk\allensdk\internal\api\behavior_ophys_api.py in get_stimulus_frame_rate(self)
     94     @memoize
     95     def get_stimulus_frame_rate(self):
---> 96         stimulus_timestamps = self.get_stimulus_timestamps()
     97         return np.round(1 / np.mean(np.diff(stimulus_timestamps)), 0)
     98 
c:\users\farzaneh.najafi\documents\analysis_codes\allensdk\allensdk\api\cache.py in wrapper(*args, **kwargs)
     73         if result is not sentinel:
     74             return result
---> 75         result = f(*args, **kwargs)
     76         cache[key] = result
     77         return result
c:\users\farzaneh.najafi\documents\analysis_codes\allensdk\allensdk\internal\api\behavior_ophys_api.py in get_stimulus_timestamps(self)
     46     def get_stimulus_timestamps(self):
     47         sync_path = self.get_sync_file()
---> 48         timestamps, _, _ = (OphysTimeAligner(sync_file=sync_path)
     49                             .corrected_stim_timestamps)
     50         return timestamps
c:\users\farzaneh.najafi\documents\analysis_codes\allensdk\allensdk\internal\brain_observatory\time_sync.py in corrected_stim_timestamps(self)
    341 
    342         photodiode_key = self._keys["photodiode"]
--> 343         delay = monitor_delay(self.dataset, timestamps, photodiode_key)
    344 
    345         return timestamps + delay, delta, delay
c:\users\farzaneh.najafi\documents\analysis_codes\allensdk\allensdk\internal\brain_observatory\time_sync.py in monitor_delay(sync_dset, stim_times, photodiode_key, transition_frame_interval, max_monitor_delay)
     73     transition_events = photodiode_events[0:len(transitions)]
     74 
---> 75     delays = transition_events - transitions
     76     delay = np.mean(delays)
     77     logging.info(f"Calculated monitor delay: {delay}. \n "
ValueError: operands could not be broadcast together with shapes (4495,) (4496,)

Environment (please complete the following information):

• OS & version: it happens on both mac and windows.
• Python version: 3.7
• AllenSDK version: latest

Additional context NA

Do you want to work on this issue? Unfortunately I won't have the time, but if I can help with better understanding and resolving the problem, please do let me know.

[yavorska-iryna]

There is a similar error with session_id 873720614.

[alexpiet]

In SDK 2.4

session = BehaviorOphysSession.from_lims(945586450)
session.ophys_timestamps # dff_frames (len=48282) is not equal to number of split timestamps (len=48253)

Cannot load metadata either

[alexpiet]

I believe this is the same issue as #1804