Slicing grid movies

[au183]

I'm trying to generate grid movies but am encountering an issue where the slice end is greater than the end frames. What I run: kpms.generate_grid_movies(results, project_dir, model_name, coordinates=coordinates, **config(),window_size=2000); Error: C:\Users\ao711.la.conda\envs\keypoint_moseq\lib\site-packages\keypoint_moseq\viz.py:1266: UserWarning:

Videos will be downscaled by a factor of 0.16 so that the grid movies are under 1920 pixels. Use max_video_size to increase or decrease this size limit.

Generating grid movies: 0%| | 0/26 [00:00<?, ?it/s]

ValueError Traceback (most recent call last) Cell In[20], line 1 ----> 1 kpms.generate_grid_movies(results, project_dir, model_name, coordinates=coordinates, **config(),window_size=2000);

File ~.conda\envs\keypoint_moseq\lib\site-packages\keypoint_moseq\viz.py:1283, in generate_grid_movies(results, project_dir, model_name, output_dir, video_dir, video_paths, rows, cols, filter_size, pre, post, min_frequency, min_duration, dot_radius, dot_color, quality, window_size, coordinates, centroids, headings, bodyparts, use_bodyparts, sampling_options, video_extension, max_video_size, skeleton, overlay_keypoints, keypoints_only, keypoints_scale, fps, plot_options, use_dims, keypoint_colormap, downsample_rate, **kwargs) 1279 # generate grid movies 1280 for syllable, instances in tqdm.tqdm( 1281 sampled_instances.items(), desc="Generating grid movies", ncols=72 1282 ): -> 1283 frames = grid_movie( 1284 instances, 1285 rows, 1286 cols, 1287 videos, 1288 centroids, 1289 headings, 1290 edges=edges, 1291 window_size=window_size, 1292 scaled_window_size=scaled_window_size, 1293 dot_color=dot_color, 1294 pre=pre, 1295 post=post, 1296 dot_radius=dot_radius, 1297 overlay_keypoints=overlay_keypoints, 1298 coordinates=coordinates, 1299 plot_options=plot_options, 1300 downsample_rate=downsample_rate, 1301 ) 1303 path = os.path.join(output_dir, f"syllable{syllable}.mp4") 1304 write_video_clip(frames, path, fps=fps, quality=quality)

File ~.conda\envs\keypoint_moseq\lib\site-packages\keypoint_moseq\viz.py:859, in grid_movie(instances, rows, cols, videos, centroids, headings, window_size, dot_color, dot_radius, pre, post, scaled_window_size, edges, overlay_keypoints, coordinates, plot_options, downsample_rate) 856 tiles = [] 857 for key, start, end in instances: 858 tiles.append( --> 859 _grid_movie_tile( 860 key, 861 start, 862 end, 863 videos, 864 centroids, 865 headings, 866 dot_color, 867 window_size, 868 scaled_window_size, 869 pre, 870 post, 871 dot_radius, 872 overlay_keypoints, 873 edges, 874 coordinates, 875 plot_options, 876 downsample_rate, 877 ) 878 ) 880 tiles = np.stack(tiles).reshape( 881 rows, cols, post + pre, scaled_window_size, scaled_window_size, 3 882 ) 883 frames = np.concatenate(np.concatenate(tiles, axis=2), axis=2)

File ~.conda\envs\keypoint_moseq\lib\site-packages\keypoint_moseq\viz.py:711, in _grid_movie_tile(key, start, end, videos, centroids, headings, dot_color, window_size, scaled_window_size, pre, post, dot_radius, overlay_keypoints, edges, coordinates, plot_options, downsample_rate) 708 tile = [] 710 if videos is not None: --> 711 frames = videos[key][ 712 (start - pre) downsample_rate : (start + post) downsample_rate 713 ][::downsample_rate] 714 c = r @ c - window_size // 2 715 M = [[np.cos(h), np.sin(h), -c[0]], [-np.sin(h), np.cos(h), -c[1]]]

File ~.conda\envs\keypoint_moseq\lib\site-packages\vidio\read.py:70, in BaseReader.getitem(self, *args, kwargs) 62 def getitem(self, *args, *kwargs) -> Union[np.ndarray, list]: 63 """Wrapper around read 64 65 Args: (...) 68 frame = reader[10] 69 """ ---> 70 return self.read(args, kwargs)

File ~.conda\envs\keypoint_moseq\lib\site-packages\vidio\read.py:107, in OpenCVReader.read(self, framenum) 102 """Read the frame indicated in framenum from disk 103 104 Uses sequential reads where possible if using OpenCV to read 105 """ 106 # does checks. if framenum is a slice, calls read recursively. In that case, just return --> 107 output = super().read(framenum) 108 if output is not None: 109 return output

File ~.conda\envs\keypoint_moseq\lib\site-packages\vidio\read.py:37, in BaseReader.read(self, framenum) 32 """Read the frame indicated in framenum from disk 33 34 Uses sequential reads where possible if using OpenCV to read 35 """ 36 if type(framenum) == slice: ---> 37 return [self.read(i) for i in self.slice_to_list(framenum)] 38 if framenum < 0 or framenum > self.nframes: 39 raise ValueError('frame number requested outside video bounds: {}'.format(framenum))

File ~.conda\envs\keypoint_moseq\lib\site-packages\vidio\read.py:44, in BaseReader.slice_to_list(self, slice_obj) 41 def slice_to_list(self, slice_obj): 42 # https://stackoverflow.com/questions/13855288/turn-slice-into-range 43 if slice_obj.stop > self.nframes: ---> 44 raise ValueError('Slice end {} > nframes {}'.format(slice_obj.stop, self.nframes)) 45 return list(range(self.nframes)[slice_obj])

ValueError: Slice end 528 > nframes 422

[calebweinreb]

Hey! Here's some steps we can take to diagnose the issue:

First let's look at the mapping between keypoint data and videos that kpms is running. Can you run the following code and confirm that the results make sense? It should print a dictionary where the keys are roughly the names of files that you loaded the keypoint coordinates from and the values are video paths.

video_dir = config()["video_dir"]
print(kpms.find_matching_videos(results.keys(), video_dir))

If the above output makes sense, then can you manually confirm that the video lengths make sense? i.e. make a key from the dictionary, print the length you could expect (print(len(coordinates[key]))) and then confirm that the corresponding video is the same length?

[au183]

Hi there, thanks for helping. As far as I the videos file names are the same as the keypoint coordinate data file. The only difference being the keypoint data having the DLC model and such tacked onto the end of the video filename. I'm totally ignorant and new to python so I apologize, but for your second suggestion of what to check, what dictionary should I reference.

[calebweinreb]

If you run the code I pasted above, then a dictionary should be printed. Does that work?

[au183]

These lines? video_dir = config()["video_dir"] print(kpms.find_matching_videos(results.keys(), video_dir)) Yes that works, and the videos match up.

This line: print(len(coordinates[key])) is not working unless I should be replacing something.

[calebweinreb]

For this like print(len(coordinates[key]))

you should replace "key" with a string corresponding to one of the keys of the dictionary that is printed by the first code chunk. Then verify that the corresponding video has the same length.

[au183]

For the first bit of code the output is: ['C:/Users/ao711.la/Desktop/BW R VIDEO ORG/moseq\G-354.4 D1 T10 R.mp4',.....] Running: print(len(coordinates['C:/Users/ao711.la/Desktop/BW R VIDEO ORG/moseq\G-354.4 D1 T10 R.mp4'])) Give the following: KeyError: 'C:/Users/ao711.la/Desktop/BW R VIDEO ORG/moseq\G-354.4 D1 T10 R.mp4'

[calebweinreb]

Oh sorry try running

video_dir = config()["video_dir"]
print(kpms.find_matching_videos(results.keys(), video_dir, as_dict=True))

to get the output as a dictionary

[au183]

Ok so I tested a few keys at random and get the same error each time:

In[35], line 1 ----> 1 print(len(coordinates['t-218.4 d3 t (8)DLC_resnet50_BW_moseq _RSApr4shuffle1_600000': 'C:/Users/ao711.la/Desktop/BW R VIDEO ORG/moseq\t-218.4 d3 t (8).mp4']))

TypeError: unhashable type: 'slice'

[calebweinreb]

Because there's two ' characters inside the key, you have to use double quotes for it to be syntactic...

print(len(coordinates["t-218.4 d3 t (8)DLC_resnet50_BW_moseq _RSApr4shuffle1_600000': 'C:/Users/ao711.la/Desktop/BW R VIDEO ORG/moseq\t-218.4 d3 t (8).mp4"]))

[au183]

Sorry about that, entering your line gives the following. Substituting keys gives the same error for other keys entered.

KeyError: "t-218.4 d3 t (8)DLC_resnet50_BW_moseq _RSApr4shuffle1_600000': 'C:/Users/ao711.la/Desktop/BW R VIDEO ORG/moseq\t-218.4 d3 t (8).mp4"

[calebweinreb]

What's the output of

print(list(coordinates.keys()))

Are the keys you're trying not there?

[au183]

I have been testing the keys that are printed. print(list(coordinates.keys())) ['G-354.4 D1 T10 RDLC_resnet50_BW_moseq _RSApr4shuffle1_600000',......]

print(kpms.find_matching_videos(results.keys(), video_dir, as_dict=True)) {'G-354.4 D1 T10 RDLC_resnet50_BW_moseq _RSApr4shuffle1_600000': 'C:/Users/ao711.la/Desktop/BW R VIDEO ORG/moseq\G-354.4 D1 T10 R.mp4',....}

[calebweinreb]

If

print(len(coordinates[key]))

gives a KeyError even when you directly copy the key from the list generated by print(list(coordinates.keys())), then there must be some kind of syntax error in how the string has been copied. I would at least expect "G-354.4 D1 T10 RDLC_resnet50_BW_moseq _RSApr4shuffle1_600000" to work since it doesn't have any problematic characters like back slashes. Anyway this seems like a basic python issue so it might be best to use chatGPT or consult with someone local who knows python and then we can proceed once you are able to check the number of frames in the coordinates arrays.

[au183]

Ok thanks anyways