LorenFrankLab / rec_to_nwb

Data Migration REC -> NWB 2.0 Service
Other
2 stars 8 forks source link

error in position_originator related to valid gaps #30

Closed jguides closed 2 years ago

jguides commented 2 years ago

When generating an nwb file, I got the following error. It looks like the code was expecting is_valid_gap to be non-empty, but it was empty in my case. Thanks in advance for any thoughts about what could be going on here.

`--------------------------------------------------------------------------- IndexError Traceback (most recent call last) /tmp/ipykernel_3384249/1204151749.py in 46 trodes_rec_export_args=trodes_rec_export_args) 47 ---> 48 content = builder.build_nwb() 49 print(content)

~/Src/rec_to_nwb/rec_to_nwb/processing/builder/raw_to_nwb_builder.py in build_nwb(self, run_preprocessing, process_mda_valid_time, process_mda_invalid_time, process_pos_valid_time, process_pos_invalid_time) 230 process_mda_invalid_time=process_mda_invalid_time, 231 process_pos_valid_time=process_pos_valid_time, --> 232 process_pos_invalid_time=process_pos_invalid_time) 233 234 logger.info('Done...\n')

~/Src/rec_to_nwb/rec_to_nwb/processing/builder/raw_to_nwb_builder.py in __build_nwb_file(self, process_mda_valid_time, process_mda_invalid_time, process_pos_valid_time, process_pos_invalid_time) 245 logger.info('Date: {}'.format(date)) 246 nwb_builder = self.get_nwb_builder(date) --> 247 content = nwb_builder.build() 248 nwb_builder.write(content) 249 if self.is_old_dataset:

~/Src/rec_to_nwb/rec_to_nwb/processing/builder/nwb_file_builder.py in build(self) 369 self.associated_files_originator.make(nwb_content) 370 --> 371 self.position_originator.make(nwb_content) 372 373 valid_map_dict = self.__build_corrupted_data_manager()

~/Src/rec_to_nwb/rec_to_nwb/processing/tools/beartype/beartype.py in func_beartyped(__beartype_func, *args, **kwargs)

~/Src/rec_to_nwb/rec_to_nwb/processing/builder/originators/position_originator.py in make(self, nwb_content) 50 zip(meters_per_pixels, pos_online_paths)): 51 position_df = self.get_position_with_corrected_timestamps( ---> 52 pos_online_path) 53 position.create_spatialseries( 54 name=f'series{series_id}',

~/Src/rec_to_nwb/rec_to_nwb/processing/builder/originators/position_originator.py in get_position_with_corrected_timestamps(pos_online_path) 92 dio_systime = np.asarray(continuous_time.loc[dio_camera_ticks]) 93 ---> 94 pause_mid_time = find_acquisition_timing_pause(dio_systime) 95 96 ptp_systime = np.asarray(camera_hwsync.HWTimestamp)

~/Src/rec_to_nwb/rec_to_nwb/processing/builder/originators/position_originator.py in find_acquisition_timing_pause(timestamps, min_duration, max_duration, n_search) 278 is_valid_gap = (timestamp_difference > min_duration) & ( 279 timestamp_difference < max_duration) --> 280 pause_start_ind = np.nonzero(is_valid_gap)[0][0] 281 pause_end_ind = pause_start_ind + 1 282 pause_mid_time = (

IndexError: index 0 is out of bounds for axis 0 with size 0

`

jguides commented 2 years ago

@edeno curious if you have any thoughts about what this could be. Thanks in advance.

edeno commented 2 years ago

The code is attempting to find the timing pause that lines up the DIO camera ticks and the camera frames. I can look at why it's not finding the gap if you tell me the location of the position file it is attempting to process

shijiegu commented 2 years ago

I have the same error in issue #27

edeno commented 2 years ago

Which file are you having the issue on? I tried it on all of molly 20220420 and had no issues

shijiegu commented 2 years ago

Thanks Eric. It was indeed molly20220420. And maybe now it is fixed. I am running it. I was just looking at Jen's error message again, I think she has not pulled the most recent code as in the new code, pos_online is renamed. @jguides Jen, please check if you have the most recent code.

jguides commented 2 years ago

Thanks both. For me, this ended up being an issue with the way one of the position files was named. After fixing this, the error did not occur.