Step 8c Stalling Before Completion

[ChamberlainSci]

I have tried two different videos and each of them stopped before reaching 100% in Step 8c. I am not sure what is wrong. One stopped at 77% and the other at 68%. The first times it did it I was able to see the first 5 rows of tracking data even though it had stalled, but this most recent time it gave an error message. I am new to python/jupyter notebook/etc.

location = lt.TrackLocation(video_dict, tracking_params)
location.to_csv(os.path.splitext(video_dict['fpath'])[0] + '_LocationOutput.csv', index=False) location.head() 68%|██████████████████████████████████████████████████████████████████▌ | 38655/56961 [59:47<28:18, 10.78it/s] total frames processed: 38654

---

KeyError Traceback (most recent call last)

in ----> 1 location = lt.TrackLocation(video_dict, tracking_params) 2 location.to_csv(os.path.splitext(video_dict['fpath'])[0] + '_LocationOutput.csv', index=False) 3 location.head() ~\Downloads\ezTrack-master\ezTrack-master\LocationTracking_Functions.py in TrackLocation(video_dict, tracking_params) 765 766 #add region of interest info --> 767 df = ROI_Location(video_dict, df) 768 769 #update scale, if known ~\Downloads\ezTrack-master\ezTrack-master\LocationTracking_Functions.py in ROI_Location(video_dict, location) 1125 #Create ROI Masks 1126 ROI_masks = {} -> 1127 for poly in range(len(video_dict['roi_stream'].data['xs'])): 1128 x = np.array(video_dict['roi_stream'].data['xs'][poly]) #x coordinates 1129 y = np.array(video_dict['roi_stream'].data['ys'][poly]) #y coordinates KeyError: 'roi_stream'

[ZachPenn]

The stopping prior to completion likely has to do with your video file type and the method of compression used during video acquisition. In particular, some videos have large numbers of incomplete frames to support smaller file sizes (see https://en.wikipedia.org/wiki/Video_compression_picture_types for a description of 'p frames'). Currently, ezTrack does not possess a good way to handle videos with these incomplete, 'p frames', and simply skips them. While all full frames can be processed, frame indexing gets really weird (because key frames 1,2,3,4,5 might have indices 1,11,21,31,41, or something like this).

It may be possible for me to implement something to handle this, but I don't have any videos like yours so I can't do any testing. If you would like to post a video here or reach out to me via email, please do. I would like to start dealing with this. Until this is sorted out, I might look into video converters that drop these frames.

Regarding your second issue, I'm guessing you specified in Cell 2 that there were regions of interest, but then did not draw them. This is why the last line of the error report says you are missing the 'roi_stream'.

[ChamberlainSci]

Good morning, Thank you for your reply!

1- I found that if I slightly trimmed the videos (in windows video player) then it saves them as mp4 rather than avi files, and this process seems to allow them to get all the way to 100% on the tracking. Hooray! I would be happy to share videos of the .avi type so you can try to improve the program. I don't know the best way to share though. Maybe I can take a short video today that will be small enough to email.

2- However, my new problem is that the location tracking is not picking up the crayfish. The location track map below shows minimal red tracking line and it is all outside the maze (1st screenshot). The crayfish was in the maze for 30 minutes, including several passes around the "lit" portions of the enclosure (2nd screenshot), so I don't know why it is not being picked up?

The initial display examples (bottom of 3rd screenshot) to confirm tracking threshold identified the crayfish when it was in the lit area. So, I don't know why it didn't track it in the full track.

3- The confirm tracking examples were not great (top of 3rd screenshot) when the crayfish was hidden in the dark areas because the program was picking up some shadows rather than the crayfish, but I figured this would be eliminated if I used the window size feature. Perhaps there is something I can do to prevent it identifying pebbles. Or maybe I should lighten the video in editing to make the dark areas less dark so the crayfish can be seen?

I appreciate any insight you can offer. Thanks!

[ZachPenn]

Glad to hear converting the video helped.

With respect to your new issue, I think your video may be too dark. It appears as though the animal will be completely invisible when in the dark arms of your maze (no contrast between the animal and the background) and ezTrack will have issues in this instance. Currently, ezTrack requires that the animal always be visible to work effectively.

That said, I might try changing the following parameters (see the screenshot):

• Reduce the value of the parameter loc_thresh from the default. This will decrease the threshold of what can be detected.
• Set method to 'dark'
• Lastly, if you use the window feature, I might play with the window_weight parameter. If the animal disappears, if this weight is too high when the animal reappears the tracking won't be able to snap back to the animal quickly.

If you are still able to change video settings (i.e. you are still collecting videos), I might increase your exposure time and gain so that the dark parts of the arena are lighter.