DLC data import issue

[jelrod14]

Hi, Just started using keypoint moseq, and am getting an error when trying to load the DLC data:

TypeError                                 Traceback (most recent call last)
Cell In[31], line 6
      3 coordinates, confidences = kpms.load_deeplabcut_results(dlc_results_directory)
      5 # format data for modeling
----> 6 data, labels = kpms.format_data(coordinates, confidences=confidences, **config())

Cell In[28], line 5, in <lambda>()
      1 import keypoint_moseq as kpms
      3 project_dir = 'demo_project'
----> 5 config = lambda: kpms.load_config(project_dir)

File ~\Anaconda3\envs\keypoint_moseq\lib\site-packages\keypoint_moseq\io.py:217, in load_config(project_dir, check_if_valid, build_indexes)
    214     config = yaml.safe_load(stream)
    216 if check_if_valid: 
--> 217     check_config_validity(config)
    219 if build_indexes:
    220     config['anterior_idxs'] = jnp.array(
    221         [config['use_bodyparts'].index(bp) for bp in config['anterior_bodyparts']])

File ~\Anaconda3\envs\keypoint_moseq\lib\site-packages\keypoint_moseq\io.py:162, in check_config_validity(config)
    157     if not bodypart in config['bodyparts']:
    158         error_messages.append(           
    159             f'ACTION REQUIRED: `use_bodyparts` contains {bodypart} '
    160             'which is not one of the options in `bodyparts`.')
--> 162 for bodypart in sum(config['skeleton'],[]):
    163     if not bodypart in config['bodyparts']:
    164         error_messages.append(
    165             f'ACTION REQUIRED: `skeleton` contains {bodypart} '
    166             'which is not one of the options in `bodyparts`.')

TypeError: 'NoneType' object is not iterable

My skeleton is set to null, and my body parts are the same in the bodypart list as they are in use_bodyparts. I think the lamda function is having issues pulling the config? Thanks!

[calebweinreb]

The skeleton should be set to [] rather than null. Although we can make null an option in the next release.

[jelrod14]

Ah interesting. Thanks for the help! It works now. I guess while I have this issue open, I'll quickly ask another issue im having. When I try to run the noise calibration step, it keeps saying that it cannot find a matching video for a file name, despite the videos being named the same as the csv files. I have about 250 videos, and all of them appear to load correctly now in the previous step. Here's the error message:

AssertionError                            Traceback (most recent call last)
Cell In[12], line 1
----> 1 kpms.noise_calibration(project_dir, coordinates, confidences, **config())

File ~\Anaconda3\envs\keypoint_moseq\lib\site-packages\keypoint_moseq\calibration.py:441, in noise_calibration(project_dir, coordinates, confidences, bodyparts, use_bodyparts, video_dir, video_extension, conf_pseudocount, verbose, **kwargs)
    438 annotations = load_annotations(project_dir)
    439 sample_keys.extend(annotations.keys())
--> 441 sample_images = load_sampled_frames(
    442     sample_keys, video_dir, video_extension=video_extension)
    444 return _noise_calibration_widget(
    445     project_dir, coordinates, confidences, sample_keys, 
    446     sample_images, annotations, bodyparts=bodyparts, **kwargs)

File ~\Anaconda3\envs\keypoint_moseq\lib\site-packages\keypoint_moseq\calibration.py:101, in load_sampled_frames(sample_keys, video_dir, video_extension)
     78 """
     79 Load sampled frames from a directory of videos.
     80 
   (...)
     98 
     99 """
    100 keys = sorted(set([k[0] for k in sample_keys]))
--> 101 videos = find_matching_videos(keys,video_dir)
    102 key_to_video = dict(zip(keys,videos))
    103 readers = {key: OpenCVReader(video) for key,video in zip(keys,videos)}

File ~\Anaconda3\envs\keypoint_moseq\lib\site-packages\keypoint_moseq\util.py:301, in find_matching_videos(keys, video_dir, as_dict, recursive, session_name_suffix, video_extension)
    298 for key in keys:
    299     matches = [v for v in videos_to_paths if \
    300                os.path.basename(key).startswith(v+session_name_suffix)]
--> 301     assert len(matches)>0, fill(f'No matching videos found for {key}')
    303     longest_match = sorted(matches, key=lambda v: len(v))[-1]
    304     video_paths.append(videos_to_paths[longest_match])

AssertionError: No matching videos found for 032921_Het_P3

[calebweinreb]

Can you provide more details? What are the names of all the videos? What are the names of the keypoint annotation files? Are they all in the same directory? What exact commands did you use for the calibration step? Also what did you put as video_dir in the config?

[jelrod14]

Names of the videos all follow a date_genotype_trial.mp4 (example: 022018_WT_P3) format. occasionally there are some that have date_genotype_identifier_trial.mp4 format (example: 032321_WT_LH_P0). The csv's are named the same as the videos. they are all in the same directory. command for calibration step: kpms.noise_calibration(project_dir, coordinates, confidences, **config()) Video directory is 'C:/Users/jelrod14/Desktop/videos2/' in config

[calebweinreb]

It's a little bit hard to debug from a distance, but here are some things to try:

First, we can check if keypoint_moseq is able to find the csv and mp4 files. What's the output when you run the following? It should find all the mp4 and csv files in the videos2 directory. If so, can you copy and paste the output or email it to calebsw@gmail.com?

from keypoint_moseq.util import list_files_with_exts

video_dir = 'C:/Users/jelrod14/Desktop/videos2/'
print(list_files_with_exts(video_dir, ['mp4']))
print(list_files_with_exts(video_dir, ['csv']))

If that didn't work, what happens when you run the following?

import glob, os
glob.glob(os.path.join(video_dir, '*'))

[jelrod14]

Just sent the output

[calebweinreb]

In the output you sent, there are 237 .csv files but only 58 .mp4 files, so it makes sense that keypoint_moseq cannot find the corresponding video for many of the .csv files. Are there mp4 files in the directory that are not being detected by list_files_with_exts?

[jelrod14]

Yes, there are mp4 files for all of the csv's in the folder. When I use the glob function, I see all of the csv's and mp4's.

Could a potential issue be that not all of these videos were ran in the exact same DLC project?

[calebweinreb]

Hmm it's odd that glob sees all the mp4s but list_files_with_exts doesn't... can you email the full list of mp4s and csvs that's output by glob?

[jelrod14]

just sent! Thanks for taking the time to help on this!

[calebweinreb]

It looks like the problem is that some of the videos ended in ".mp4" whereas others ended in ".Mp4". I just rewrote list_files_with_exts so that now it can handle mixed-case file extensions. Can you try upgrading to the dev branch and let me know if that fixes the issue? You can upgrade by running the following in the keypoint_moseq conda environment.

pip install -U git+https://github.com/dattalab/keypoint-moseq@dev

[jelrod14]

Getting this error now when I try to import keypoint_moseq

ImportError: cannot import name 'check_for_nans' from 'jax_moseq.utils' (C:\Users\jelrod14\Anaconda3\envs\keypoint_moseq\lib\site-packages\jax_moseq\utils\__init__.py)

[calebweinreb]

Sorry you'll need to update jax_moseq too:

pip install -U jax-moseq

[jelrod14]

Everything is working now! Thank you so much!