Grid Movies & Syllable Dendrogram After Applying a Network to New Data

[mbcooper97]

I am using out cluster (A100 GPU + Linux HPC Desktop). I trained a model, then applied that model on new data, and when I tried to generate grid movies + dendrograms of the new and original data I get the following error:

>>> kpms.generate_trajectory_plots(coordinates, results, project_dir, model_name, **config()) Saving trajectory plots to /home/mbullock/Desktop/Glacier_KPMS/2024_02_09-13_20_39/trajectory_plots Traceback (most recent call last): File "", line 1, in File "/home/mbullock/.conda/envs/keypoint_moseq/lib/python3.9/site-packages/keypoint_moseq/viz.py", line 1712, in generate_trajectory_plots typical_trajectories = get_typical_trajectories( File "/home/mbullock/.conda/envs/keypoint_moseq/lib/python3.9/site-packages/keypoint_moseq/util.py", line 1139, in get_typical_trajectories sampled_instances = sample_instances( File "/home/mbullock/.conda/envs/keypoint_moseq/lib/python3.9/site-packages/keypoint_moseq/util.py", line 633, in sample_instances trajectories = { File "/home/mbullock/.conda/envs/keypoint_moseq/lib/python3.9/site-packages/keypoint_moseq/util.py", line 634, in syllable: get_instance_trajectories( File "/home/mbullock/.conda/envs/keypoint_moseq/lib/python3.9/site-packages/keypoint_moseq/util.py", line 529, in get_instance_trajectories [ File "/home/mbullock/.conda/envs/keypoint_moseq/lib/python3.9/site-packages/keypoint_moseq/util.py", line 530, in coordinates[key][s - pre : s + post] KeyError: '2359-07072023114228DLC_resnet50_GlacierJan22shuffle1_1001000'

[calebweinreb]

Does coordinates contain both the new and old data?

[calebweinreb]

(closing because of inactivity)

[andlock97]

during the tutorial, at the step where you can apply to new data I believe there should be a line added to update the coordinates file to contain both the old key points and the new ones, unless I missed somewhere else it can be updated `new_data = 'new_data_file.predictions.analysis.h5'

coordinates, confidences, bodyparts = kpms.load_keypoints(new_data, 'sleap')

original_data = 'original_data_file.predictions.analysis.h5'

coordinates2, confidences2, bodyparts2 = kpms.load_keypoints(original_data, 'sleap')

coordinates.update(coordinates2)`

[calebweinreb]

That line would be useful if you wanted to simultaneously apply the model to new data AND re-apply it to the the original data. That's not the use case documented in the tutorial, but definitely a reasonable thing to do!

[andlock97]

when applying the model to new data, does apply_model refit the original model? Then to get the trajectory and grid movie plots for each set of data we should reload the coordinates for each set and also pass their respective part of the results.h5 file?

[calebweinreb]

apply_model doesn't refit the original model, so in theory you don't need to reapply to the original data. Although anecdotally it seems that the results do shift slightly so using apply_model to re-infer syllables for the original data isn't a bad idea.