calibration stalls, or only shows keypoints in first frame

[csoneson]

Hi, thanks for putting together a nice package. I have a question related to the noise calibration, which I was hoping you might be willing to help me with. In short, the keypoints (and the 'x' for the selected bodypart) are only displayed in the first frame appearing in the widget, not in the subsequent ones. If this is intentional, then all is good (in that case, perhaps this could be mentioned in the tutorial, as the frame shown there has the keypoints indicated and it mentions that an 'x' should appear). If not, please see below for more details.

---

As a test, I downloaded the files videos/21_11_8_one_mouse.top.irDLC_resnet50_moseq_exampleAug21shuffle1_500000.h5, videos/21_11_8_one_mouse.top.ir.mp4 and config.yaml from the dlc_project example folder, and placed them in a folder named example_data_moseq. I then ran the following code (pretty much direct from the tutorial) in JupyterLab:

import keypoint_moseq as kpms

project_dir = 'example_data_moseq_output'
config = lambda: kpms.load_config(project_dir)

dlc_config = 'example_data_moseq/config.yaml'
kpms.setup_project(project_dir, deeplabcut_config=dlc_config, overwrite=True)

kpms.update_config(
    project_dir,
    video_dir='example_data_moseq',
    anterior_bodyparts=['nose'],
    posterior_bodyparts=['spine4'],
    use_bodyparts=[
        'spine4', 'spine3', 'spine2', 'spine1',
        'head', 'nose', 'right ear', 'left ear'])

# load data (e.g. from DeepLabCut)
keypoint_data_path = 'example_data_moseq'
coordinates, confidences, bodyparts = kpms.load_keypoints(keypoint_data_path, 'deeplabcut')

# format data for modeling
data, metadata = kpms.format_data(coordinates, confidences, **config())

# calibration
kpms.noise_calibration(project_dir, coordinates, confidences, **config())

This brings up the expected widget, and the keypoints are located in the right place. In keypoint-moseq v0.3.2, if I go to the next frame (or any subsequent one), no keypoints are displayed (I zoomed out to see if they were in the wrong location, but they were not there at all). The same also happens with our own data. With keypoint-moseq v0.4.2, I am not able to click in the image at all (the kernel says that it's 'busy' for a short while, but no point is added to the scatter plot), and also nothing happens if I attempt to click on the arrow to go to the next frame (perhaps this is a different issue).

I am on Linux, and I installed keypoint-moseq (CPU version) via pip, following the instructions. I also tried the conda installation on a mac, and also there (with v0.4.2) the keypoints are only shown in the first frame). JupyterLab is version 4.0.10.

Any pointers to what the issue might be, or things to try, would be appreciated. Many thanks in advance!

[Lauraschwarz]

the very same thing happens to me as well.

i am using windows with the keypoint moseq GPU version (installed using conda on 12/01/2024) also folling the instructions on the tutorial page. (kpms version: 0.4.2 // jupyterlab version: 4.0.10)

[calebweinreb]

Hi, I just released a new version (0.4.3) that should address this. Just make sure you run the widget from jupyter lab, and that jupyter is launched from the keypoint_moseq environment.

[csoneson]

@calebweinreb Thanks for looking into this. I installed the new version and ran the same code as before - unfortunately, now I don't see the widget at all (there are no error messages indicating that anything went wrong). Any idea what the reason might be?

[calebweinreb]

What OS is that error from? Also what versions of jupyter, panel, bokeh and holoviews do you have?

[anayapouget]

@calebweinreb I am getting the same issue as @csoneson. I am using Windows and the cpu version of kpms 0.4.3. For the packages you specified, my versions are: JupyterLab: 4.0.10 Panel: 1.3.6 Bokeh: 3.3.3 Holoviews: 1.18.1

[csoneson]

I'm on Linux, with versions:

jupyterlab                4.0.10
jupyter-client            8.6.0
jupyter-core              5.7.1
panel                     1.3.6
bokeh                     3.3.3
holoviews                 1.18.1

[versey-sherry]

Did you conda activate the keypoint-moseq environment before you use jupyterlab?

[anayapouget]

@versey-sherry yes I did

[versey-sherry]

Could you try pip install notebook to install Jupyter notebook and see if it works in Jupyter notebook?

[anayapouget]

Actually I think notebook is already installed in the kpms environment but yea I just tried running the code in Jupyter notebook and it didn't make a difference unfortunately...

[versey-sherry]

Could you try conda update jupyterlab to update Juypter Lab 4.0.11 and see if things work? I wasn't able to reproduce your issue on my computer. Is it possible that your computer is low in memory?

[anayapouget]

Hm it works now - I tried a lot of different things because conda update jupyterlab wasn't working, not really sure why the pane started appearing. Although if I click on the arrow to move to a new image and then click back to the previous one, the skeleton appears as normal but the image itself doesn't appear again... It's fine since clicking back isn't necessarily an indispensible feature but it's just a bit strange. I've not had issues with the noise calibration widget before updating to the newest version of moseq, weird that it is so complicated to get working on my computer now!

[csoneson]

After reinstalling keypoint-moseq from conda today, it appears to work reasonably well for me too (reinstalling via pip yesterday did not help). Sometimes it seems to require reloading JupyterLab (even if it was just started) to have the plots show up. I should mention that in those cases, I also have issues when generating a 'simple' holoviews plot (again, the plot doesn't display), so my guess is that it's somehow related to that extension.

[versey-sherry]

I am glad things are working for you now! I suspect there are a few issues here: 1. the packages are not loaded correctly after updating 2. the memory use become higher so the widget won't load.

If both of you have no more questions, I will close this issue.

[versey-sherry]

Tested on Windows, Linux and MacOS, calibration widget worked fine.