Issue downloading the jupyter extensions in python.

[rdemann]

Hi, After installing the most recent version of keypoint_moseq with the latest version of jax (I had compatibility issues with jax[cpu]==0.3.22). I tried to download the jupyter extensions from the Terminal.

The workstation I am using is a Dell Inc. Precision 5820 Tower with 64GB of RAM, Intel® Xeon(R) W-2104 CPU @ 3.20GHz × 4 processor, AMD® Radeon pro wx 2100 graphics card, and 1TB disk capacity running Ubuntu 22.04.2 LTS.

jupyter nbextension install --py jupyter_nbextensions_configurator --sys-prefix

(keypoint_moseq) canliu@thebeast:~$ jupyter nbextension install --py jupyter_nbextensions_configurator --sys-prefix Traceback (most recent call last): File "/usr/bin/jupyter-nbextension", line 33, in sys.exit(load_entry_point('notebook==6.4.8', 'console_scripts', 'jupyter-nbextension')()) File "/usr/lib/python3/dist-packages/jupyter_core/application.py", line 264, in launch_instance return super(JupyterApp, cls).launch_instance(argv=argv, **kwargs) File "/usr/lib/python3/dist-packages/traitlets/config/application.py", line 846, in launch_instance app.start() File "/usr/lib/python3/dist-packages/notebook/nbextensions.py", line 980, in start super().start() File "/usr/lib/python3/dist-packages/jupyter_core/application.py", line 253, in start self.subapp.start() File "/usr/lib/python3/dist-packages/notebook/nbextensions.py", line 708, in start self.install_extensions() File "/usr/lib/python3/dist-packages/notebook/nbextensions.py", line 679, in install_extensions full_dests = install(self.extra_args[0], File "/usr/lib/python3/dist-packages/notebook/nbextensions.py", line 203, in install_nbextension_python m, nbexts = _get_nbextension_metadata(module) File "/usr/lib/python3/dist-packages/notebook/nbextensions.py", line 1114, in _get_nbextension_metadata m = import_item(module) File "/usr/lib/python3/dist-packages/traitlets/utils/importstring.py", line 38, in import_item return import(parts[0]) ModuleNotFoundError: No module named 'jupyter_nbextensions_configurator'

jupyter nbextension enable --py --sys-prefix widgetsnbextension

(keypoint_moseq) canliu@thebeast:~$ jupyter nbextension enable --py --sys-prefix widgetsnbextension Enabling notebook extension jupyter-js-widgets/extension... Traceback (most recent call last): File "/usr/bin/jupyter-nbextension", line 33, in sys.exit(load_entry_point('notebook==6.4.8', 'console_scripts', 'jupyter-nbextension')()) File "/usr/lib/python3/dist-packages/jupyter_core/application.py", line 264, in launch_instance return super(JupyterApp, cls).launch_instance(argv=argv, **kwargs) File "/usr/lib/python3/dist-packages/traitlets/config/application.py", line 846, in launch_instance app.start() File "/usr/lib/python3/dist-packages/notebook/nbextensions.py", line 980, in start super().start() File "/usr/lib/python3/dist-packages/jupyter_core/application.py", line 253, in start self.subapp.start() File "/usr/lib/python3/dist-packages/notebook/nbextensions.py", line 888, in start self.toggle_nbextension_python(self.extra_args[0]) File "/usr/lib/python3/dist-packages/notebook/nbextensions.py", line 861, in toggle_nbextension_python return toggle(module, File "/usr/lib/python3/dist-packages/notebook/nbextensions.py", line 474, in enable_nbextension_python return _set_nbextension_state_python(True, module, user, sys_prefix, File "/usr/lib/python3/dist-packages/notebook/nbextensions.py", line 373, in _set_nbextension_state_python return [_set_nbextension_state(section=nbext["section"], File "/usr/lib/python3/dist-packages/notebook/nbextensions.py", line 373, in return [_set_nbextension_state(section=nbext["section"], File "/usr/lib/python3/dist-packages/notebook/nbextensions.py", line 343, in _set_nbextension_state cm.update(section, {"load_extensions": {require: state}}) File "/usr/lib/python3/dist-packages/notebook/config_manager.py", line 131, in update self.set(section_name, data) File "/usr/lib/python3/dist-packages/notebook/config_manager.py", line 108, in set self.ensure_config_dir_exists() File "/usr/lib/python3/dist-packages/notebook/config_manager.py", line 65, in ensure_config_dir_exists os.makedirs(self.config_dir, 0o755) File "/usr/lib/python3.10/os.py", line 215, in makedirs makedirs(head, exist_ok=exist_ok) File "/usr/lib/python3.10/os.py", line 215, in makedirs makedirs(head, exist_ok=exist_ok) File "/usr/lib/python3.10/os.py", line 225, in makedirs mkdir(name, mode) PermissionError: [Errno 13] Permission denied: '/usr/etc'

jupyter nbextension enable --py --sys-prefix --py qgrid

(keypoint_moseq) canliu@thebeast:~$ jupyter nbextension enable --py --sys-prefix --py qgrid Traceback (most recent call last): File "/usr/bin/jupyter-nbextension", line 33, in sys.exit(load_entry_point('notebook==6.4.8', 'console_scripts', 'jupyter-nbextension')()) File "/usr/lib/python3/dist-packages/jupyter_core/application.py", line 264, in launch_instance return super(JupyterApp, cls).launch_instance(argv=argv, **kwargs) File "/usr/lib/python3/dist-packages/traitlets/config/application.py", line 846, in launch_instance app.start() File "/usr/lib/python3/dist-packages/notebook/nbextensions.py", line 980, in start super().start() File "/usr/lib/python3/dist-packages/jupyter_core/application.py", line 253, in start self.subapp.start() File "/usr/lib/python3/dist-packages/notebook/nbextensions.py", line 888, in start self.toggle_nbextension_python(self.extra_args[0]) File "/usr/lib/python3/dist-packages/notebook/nbextensions.py", line 861, in toggle_nbextension_python return toggle(module, File "/usr/lib/python3/dist-packages/notebook/nbextensions.py", line 474, in enable_nbextension_python return _set_nbextension_state_python(True, module, user, sys_prefix, File "/usr/lib/python3/dist-packages/notebook/nbextensions.py", line 372, in _set_nbextension_state_python m, nbexts = _get_nbextension_metadata(module) File "/usr/lib/python3/dist-packages/notebook/nbextensions.py", line 1114, in _get_nbextension_metadata m = import_item(module) File "/usr/lib/python3/dist-packages/traitlets/utils/importstring.py", line 38, in import_item return import(parts[0]) ModuleNotFoundError: No module named 'qgrid'

jupyter nbextension install --sys-prefix --symlink --py jupyter_bokeh

(keypoint_moseq) canliu@thebeast:~$ jupyter nbextension install --sys-prefix --symlink --py jupyter_bokeh Traceback (most recent call last): File "/usr/bin/jupyter-nbextension", line 33, in sys.exit(load_entry_point('notebook==6.4.8', 'console_scripts', 'jupyter-nbextension')()) File "/usr/lib/python3/dist-packages/jupyter_core/application.py", line 264, in launch_instance return super(JupyterApp, cls).launch_instance(argv=argv, **kwargs) File "/usr/lib/python3/dist-packages/traitlets/config/application.py", line 846, in launch_instance app.start() File "/usr/lib/python3/dist-packages/notebook/nbextensions.py", line 980, in start super().start() File "/usr/lib/python3/dist-packages/jupyter_core/application.py", line 253, in start self.subapp.start() File "/usr/lib/python3/dist-packages/notebook/nbextensions.py", line 708, in start self.install_extensions() File "/usr/lib/python3/dist-packages/notebook/nbextensions.py", line 679, in install_extensions full_dests = install(self.extra_args[0], File "/usr/lib/python3/dist-packages/notebook/nbextensions.py", line 203, in install_nbextension_python m, nbexts = _get_nbextension_metadata(module) File "/usr/lib/python3/dist-packages/notebook/nbextensions.py", line 1114, in _get_nbextension_metadata m = import_item(module) File "/usr/lib/python3/dist-packages/traitlets/utils/importstring.py", line 38, in import_item return import(parts[0]) ModuleNotFoundError: No module named 'jupyter_bokeh'

jupyter nbextension enable jupyter_bokeh --py --sys-prefix

(keypoint_moseq) canliu@thebeast:~$ jupyter nbextension enable jupyter_bokeh --py --sys-prefix Traceback (most recent call last): File "/usr/bin/jupyter-nbextension", line 33, in sys.exit(load_entry_point('notebook==6.4.8', 'console_scripts', 'jupyter-nbextension')()) File "/usr/lib/python3/dist-packages/jupyter_core/application.py", line 264, in launch_instance return super(JupyterApp, cls).launch_instance(argv=argv, **kwargs) File "/usr/lib/python3/dist-packages/traitlets/config/application.py", line 846, in launch_instance app.start() File "/usr/lib/python3/dist-packages/notebook/nbextensions.py", line 980, in start super().start() File "/usr/lib/python3/dist-packages/jupyter_core/application.py", line 253, in start self.subapp.start() File "/usr/lib/python3/dist-packages/notebook/nbextensions.py", line 888, in start self.toggle_nbextension_python(self.extra_args[0]) File "/usr/lib/python3/dist-packages/notebook/nbextensions.py", line 861, in toggle_nbextension_python return toggle(module, File "/usr/lib/python3/dist-packages/notebook/nbextensions.py", line 474, in enable_nbextension_python return _set_nbextension_state_python(True, module, user, sys_prefix, File "/usr/lib/python3/dist-packages/notebook/nbextensions.py", line 372, in _set_nbextension_state_python m, nbexts = _get_nbextension_metadata(module) File "/usr/lib/python3/dist-packages/notebook/nbextensions.py", line 1114, in _get_nbextension_metadata m = import_item(module) File "/usr/lib/python3/dist-packages/traitlets/utils/importstring.py", line 38, in import_item return import(parts[0]) ModuleNotFoundError: No module named 'jupyter_bokeh'

[versey-sherry]

Could you try in your Terminal

pip install jupyter_bokeh
pip install jupyter_nbextensions_configurator
pip install qgrid

[rdemann]

If I run:

import keypoint_moseq as kpms

project_dir = 'demo_project' config = lambda: kpms.load_config(project_dir)

I get:

---

AttributeError Traceback (most recent call last) Cell In[1], line 1 ----> 1 import keypoint_moseq as kpms 3 project_dir = 'demo_project' 4 config = lambda: kpms.load_config(project_dir)

File ~/anaconda3/envs/keypoint_moseq/lib/python3.9/site-packages/keypoint_moseq/init.py:11 7 import warnings 9 warnings.formatwarning = lambda msg, a: str(msg) ---> 11 from .io import 12 from .viz import 13 from .util import

File ~/anaconda3/envs/keypoint_moseq/lib/python3.9/site-packages/keypoint_moseq/io.py:16 13 from pynwb import NWBHDF5IO 14 from ndx_pose import PoseEstimation ---> 16 from keypoint_moseq.util import list_files_with_exts, check_nan_proportions 17 from jax_moseq.utils import get_frequencies, unbatch 20 def _build_yaml(sections, comments):

File ~/anaconda3/envs/keypoint_moseq/lib/python3.9/site-packages/keypoint_moseq/util.py:15 13 from sklearn.neighbors import NearestNeighbors 14 from scipy.spatial.distance import pdist, squareform ---> 15 from jax_moseq.models.keypoint_slds import inverse_rigid_transform 16 from jax_moseq.utils import get_frequencies, batch 18 na = jnp.newaxis

File ~/anaconda3/envs/keypoint_moseq/lib/python3.9/site-packages/jax_moseq/init.py:2 1 import jax ----> 2 from . import models 3 from . import utils 4 from . import _version

File ~/anaconda3/envs/keypoint_moseq/lib/python3.9/site-packages/jax_moseq/models/init.py:1 ----> 1 from . import arhmm 2 from . import slds 3 from . import keypoint_slds

File ~/anaconda3/envs/keypoint_moseq/lib/python3.9/site-packages/jax_moseq/models/arhmm/init.py:1 ----> 1 from jax_moseq.models.arhmm.initialize import 2 from jax_moseq.models.arhmm.gibbs import 3 from jax_moseq.models.arhmm.log_prob import *

File ~/anaconda3/envs/keypoint_moseq/lib/python3.9/site-packages/jax_moseq/models/arhmm/initialize.py:7 5 from jax_moseq.utils import device_put_as_scalar, check_precision 6 from jax_moseq.utils.transitions import init_hdp_transitions ----> 7 from jax_moseq.utils.distributions import sample_mniw 9 from jax_moseq.models.arhmm.gibbs import resample_discrete_stateseqs 11 na = jnp.newaxis

File ~/anaconda3/envs/keypoint_moseq/lib/python3.9/site-packages/jax_moseq/utils/distributions.py:3 1 import jax, jax.numpy as jnp, jax.random as jr 2 import tensorflow_probability.substrates.jax.distributions as tfd ----> 3 from dynamax.hidden_markov_model.inference import hmm_posterior_sample 4 from jax_moseq.utils import convert_data_precision 6 na = jnp.newaxis

File ~/anaconda3/envs/keypoint_moseq/lib/python3.9/site-packages/dynamax/hidden_markov_model/init.py:1 ----> 1 from dynamax.hidden_markov_model.models.abstractions import HMM, HMMEmissions, HMMInitialState, HMMTransitions, HMMParameterSet, HMMPropertySet 2 from dynamax.hidden_markov_model.models.arhmm import LinearAutoregressiveHMM 3 from dynamax.hidden_markov_model.models.bernoulli_hmm import BernoulliHMM

File ~/anaconda3/envs/keypoint_moseq/lib/python3.9/site-packages/dynamax/hidden_markov_model/models/abstractions.py:2 1 from abc import abstractmethod, ABC ----> 2 from dynamax.ssm import SSM 3 from dynamax.types import Scalar 4 from dynamax.parameters import to_unconstrained, from_unconstrained

File ~/anaconda3/envs/keypoint_moseq/lib/python3.9/site-packages/dynamax/ssm.py:10 8 from jax.tree_util import tree_map 9 from jaxtyping import Float, Array, PyTree ---> 10 import optax 11 from tensorflow_probability.substrates.jax import distributions as tfd 12 from typing import Optional, Union, Tuple, Any

File ~/anaconda3/envs/keypoint_moseq/lib/python3.9/site-packages/optax/init.py:17 1 # Copyright 2019 DeepMind Technologies Limited. All Rights Reserved. 2 # 3 # Licensed under the Apache License, Version 2.0 (the "License"); (...) 13 # limitations under the License. 14 # ============================================================================== 15 """Optax: composable gradient processing and optimization, in JAX.""" ---> 17 from optax import contrib 18 from optax import experimental 19 from optax._src.alias import adabelief

File ~/anaconda3/envs/keypoint_moseq/lib/python3.9/site-packages/optax/contrib/init.py:17 1 # Copyright 2021 DeepMind Technologies Limited. All Rights Reserved. 2 # 3 # Licensed under the Apache License, Version 2.0 (the "License"); (...) 13 # limitations under the License. 14 # ============================================================================== 15 """Contributed optimizers in Optax.""" ---> 17 from optax._src.contrib.mechanic import MechanicState 18 from optax._src.contrib.mechanic import mechanize

File ~/anaconda3/envs/keypoint_moseq/lib/python3.9/site-packages/optax/_src/contrib/mechanic.py:34 31 import operator 32 from typing import NamedTuple, Optional, Tuple ---> 34 import chex 35 import jax 36 import jax.numpy as jnp

File ~/anaconda3/envs/keypoint_moseq/lib/python3.9/site-packages/chex/init.py:17 1 # Copyright 2020 DeepMind Technologies Limited. All Rights Reserved. 2 # 3 # Licensed under the Apache License, Version 2.0 (the "License"); (...) 13 # limitations under the License. 14 # ============================================================================== 15 """Chex: Testing made fun, in JAX!""" ---> 17 from chex._src.asserts import assert_axis_dimension 18 from chex._src.asserts import assert_axis_dimension_comparator 19 from chex._src.asserts import assert_axis_dimension_gt

File ~/anaconda3/envs/keypoint_moseq/lib/python3.9/site-packages/chex/_src/asserts.py:26 23 import unittest 24 from unittest import mock ---> 26 from chex._src import asserts_internal as _ai 27 from chex._src import pytypes 28 import jax

File ~/anaconda3/envs/keypoint_moseq/lib/python3.9/site-packages/chex/_src/asserts_internal.py:34 31 from typing import Any, Sequence, Union, Callable, Optional, Set, Tuple, Type 33 from absl import logging ---> 34 from chex._src import pytypes 35 import jax 36 from jax.experimental import checkify

File ~/anaconda3/envs/keypoint_moseq/lib/python3.9/site-packages/chex/_src/pytypes.py:25 23 ArrayBatched = jax.interpreters.batching.BatchTracer 24 ArrayNumpy = np.ndarray ---> 25 ArraySharded = jax.interpreters.pxla.ShardedDeviceArray 26 # For instance checking, use isinstance(x, jax.Array). 27 if hasattr(jax, 'Array'):

AttributeError: module 'jax.interpreters.pxla' has no attribute 'ShardedDeviceArray'

[rdemann]

Could you try in your Terminal

pip install jupyter_bokeh
pip install jupyter_nbextensions_configurator
pip install qgrid

I still get the same errors when trying to run the original code again...

[calebweinreb]

Hi, thanks for opening an issue.

Jupyter issue

For the jupyter issue, first can you confirm that you ran

pip install jupyter_nbextensions_configurator

in the keypoint_moseq virtual env? And after that

jupyter nbextension install --py jupyter_nbextensions_configurator --sys-prefix` 

still produced an error? If so, we saw the same issue with Windows and fixed it using conda install...

conda install -c conda-forge jupyter_contrib_nbextensions

Does that help?

Jax issue

For future reference, it is best to open a separate issue for problems unrelated to the current issue, but happy to answer here quickly!

I think the problem is that you have a Radeon pro wx 2100 graphics card, whereas JAX with cuda support only works for NVIDIA cards.

[rdemann]

Thank you so much for the reply!

I could fix the jupyter issue. I think I got the errors due to dependency issues. downgrading some of my modules helped.

Regarding the jax issue: I got the recommended jax driver to run which ended up fixing my issues.