Error when openning the viewer

[dvdmc]

I come from trying to follow the nerfstudio tutorial. I also checked that the error happens following the basic example here. I can execute the example python ./examples/02_gui.py, but when I open the browser I start getting the following error:

connection handler failed
Traceback (most recent call last):
  File "/media/david/miniconda3/envs/nerfstudio/lib/python3.8/site-packages/websockets/legacy/server.py", line 236, in handler
    await self.ws_handler(self)
  File "/media/david/miniconda3/envs/nerfstudio/lib/python3.8/site-packages/viser/infra/_infra.py", line 239, in serve
    await asyncio.gather(
  File "/media/david/miniconda3/envs/nerfstudio/lib/python3.8/site-packages/viser/infra/_infra.py", line 386, in _consumer
    message = message_class.deserialize(raw)
  File "/media/david/miniconda3/envs/nerfstudio/lib/python3.8/site-packages/viser/infra/_messages.py", line 99, in deserialize
    message_type = cls._subclass_from_type_string()[cast(str, mapping.pop("type"))]
KeyError: 'type'
^CTraceback (most recent call last):
  File "./examples/02_gui.py", line 96, in <module>
    main()
  File "./examples/02_gui.py", line 92, in main
    time.sleep(0.01)
KeyboardInterrupt
Task was destroyed but it is pending!

[brentyi]

Hi! If you're trying to run a viser example, can you make sure you're not using the v1 Nerfstudio viewer URL? Instead of navigating to https://viewer.nerf.studio/, you should just go to http://localhost:8080 in your browser directly.

[dvdmc]

Hi! Yes, that's what I was doing. I think it is a viser issue, maybe some module incompatibility?

[brentyi]

I unfortunately haven't been able to reproduce this; would you be able to go into the line that's giving the error and print out what mapping is?

Like:

if "type" not in mapping:
    print(mapping)

[dvdmc]

Here is mapping:

{b'type': b'ViewerCameraMessage', b'wxyz': [-0.17591989660616114, 0.33985114297998736, 0.8204732385702833, -0.424708200277867], b'position': [3, 3.000000000000001, 2.9999999999999996], b'aspect': 0.9676425269645609, b'fov': 1.3089969389957472, b'look_at': [0, 0, 0], b'up_direction': [0, 2.220446049250313e-16, 1]}

It comes from msgpck. My version form pip is 1.0.7. The required version from the project file is >=1.0.0.

Here is the code as a reference in file _messages.py:

     @classmethod
     def deserialize(cls, message: bytes) -> Message:
         """Convert bytes into a Python Message object."""
         mapping = msgpack.unpackb(message)
         print(mapping)
         # msgpack deserializes to lists by default, but all of our annotations use
         # tuples.
         mapping = {
             k: tuple(v) if isinstance(v, list) else v for k, v in mapping.items()
          }
          message_type = cls._subclass_from_type_string()[cast(str, mapping.pop("type"))]
          return message_type(**mapping)  # type: ignore

It's late where I am from but I will continue tomorrow. Thanks for your help so far :)

[brentyi]

Thanks! That's really weird.

It seems like the problem is likely that the mapping key types are being deserialized as bytes instead of strings, but I haven't been able to reproduce this. I'm also on msgpack 1.0.7 (and websockets 12.0, and Python 3.8).

Maybe you could share the outputs of pip freeze?

[dvdmc]

I have attached the pip freeze below. My python global environment was not clean before creating the specific environment so there are many additional libraries. Can confirm that Python is 3.8 and websockets 12.0

absl-py==1.1.0
actionlib==1.14.0
addict==2.4.0
airsim==1.8.1
alabaster==0.7.12
albumentations==1.3.0
ansi2html==1.8.0
antlr4-python3-runtime==4.9.3
anyio==3.6.1
appdirs==1.4.4
argon2-cffi==21.3.0
argon2-cffi-bindings==21.2.0
asttokens==2.0.5
astunparse==1.6.3
async-generator==1.10
attrs==23.1.0
autopep8==1.7.0
av==10.0.0
Babel==2.10.1
backcall==0.2.0
beautifulsoup4==4.11.1
bidict==0.22.1
black==22.3.0
bleach==5.0.0
blinker==1.7.0
bondpy==1.8.6
cachetools==5.2.0
casadi==3.5.5
catkin==0.8.10
certifi==2023.7.22
cffi==1.15.0
chardet==5.2.0
charset-normalizer==3.2.0
click==8.1.3
cloudpickle==2.1.0
cmake==3.27.7
colorlog==6.7.0
comet-ml==3.35.2
comm==0.2.0
ConfigArgParse==1.7
configobj==5.0.8
contourpy==1.1.1
controller-manager-msgs==0.19.6
cryptography==38.0.0
curio==1.5
cycler==0.11.0
dash==2.14.1
dash-core-components==2.0.0
dash-html-components==2.0.0
dash-table==5.0.0
debugpy==1.6.0
decorator==5.1.1
defusedxml==0.7.1
deprecation==2.1.0
descartes==1.1.0
detectron2 @ git+https://github.com/facebookresearch/detectron2.git@224cd2318fdb45b5e22bbb861ee9711ee52c8b75
diagnostic-updater==1.11.0
do-mpc==4.3.3
docker-pycreds==0.4.0
docstring-parser==0.15
dulwich==0.21.6
dynamic-reconfigure==1.7.3
efficientnet-pytorch==0.7.1
embreex==2.17.7.post3
entrypoints==0.4
everett==3.1.0
executing==0.8.3
fairscale==0.4.6
fastjsonschema==2.15.3
filelock==3.13.1
fire==0.5.0
Flask==3.0.0
flatbuffers==1.12
fonttools==4.33.3
future==0.18.2
fvcore==0.1.5.post20220512
gast==0.4.0
gdown==4.7.1
gencpp==0.7.0
geneus==3.0.0
genlisp==0.4.18
genmsg==0.6.0
gennodejs==2.0.2
genpy==0.6.15
gitdb==4.0.10
GitPython==3.1.32
google-auth==2.23.4
google-auth-oauthlib==1.0.0
google-pasta==0.2.0
grpcio==1.59.2
h11==0.14.0
h5py==3.7.0
hydra-core==1.2.0
idna==3.4
imageio==2.32.0
imagesize==1.3.0
imgaug==0.4.0
importlib-metadata==6.8.0
importlib-resources==5.7.1
iniconfig==1.1.1
iopath==0.1.9
ipykernel==6.13.0
ipyparallel==8.4.1
ipython==8.4.0
ipython-genutils==0.2.0
ipywidgets==8.1.1
itsdangerous==2.1.2
jaxtyping==0.2.19
jedi==0.18.1
Jinja2==3.1.2
joblib==1.1.0
json5==0.9.8
jsonschema==4.5.1
jupyter-client==7.3.1
jupyter-core==4.10.0
jupyter-packaging==0.12.1
jupyter-server==1.17.0
jupyterlab==3.4.2
jupyterlab-pygments==0.2.2
jupyterlab-server==2.14.0
jupyterlab-widgets==3.0.9
keras==2.9.0
Keras-Preprocessing==1.1.2
kiwisolver==1.4.4
kobuki_testsuite==0.7.6
libclang==14.0.1
lightning-utilities==0.9.0
lit==17.0.4
lpips==0.1.4
lxml==4.9.3
mapbox-earcut==1.0.1
Markdown==3.3.7
markdown-it-py==3.0.0
MarkupSafe==2.1.1
matplotlib==3.5.3
matplotlib-inline==0.1.3
mdurl==0.1.2
mediapy==1.1.9
message-filters==1.16.0
mistune==0.8.4
mpmath==1.3.0
msgpack==1.0.7
msgpack-numpy==0.4.8
msgpack-python==0.5.6
msgpack-rpc-python==0.4.1
munch==2.5.0
mypy-extensions==0.4.3
nbclassic==0.3.7
nbclient==0.6.3
nbconvert==6.5.0
nbformat==5.7.0
nerfacc==0.5.2
nerfstudio==0.3.4
nest-asyncio==1.5.5
networkx==2.8.4
ninja==1.11.1.1
nodeenv==1.8.0
notebook==6.4.11
notebook-shim==0.1.0
numpy==1.22.4
nuscenes-devkit==1.1.11
oauthlib==3.2.2
omegaconf==2.2.2
open3d==0.17.0
opencv-contrib-python==4.6.0.66
opencv-python==4.6.0.66
opencv-python-headless==4.7.0.68
opt-einsum==3.3.0
outcome==1.2.0
packaging==21.3
pandas==1.4.2
pandocfilters==1.5.0
parso==0.8.3
pathspec==0.9.0
pathtools==0.1.2
pexpect==4.8.0
pickleshare==0.7.5
Pillow==10.0.1
platformdirs==3.11.0
plotly==5.18.0
pluggy==1.0.0
portalocker==2.4.0
pr2_power_board==1.1.10
pretrainedmodels==0.7.4
prometheus-client==0.14.1
prompt-toolkit==3.0.29
protobuf==3.20.3
psutil==5.9.6
ptyprocess==0.7.0
pure-eval==0.2.2
py==1.11.0
py-cpuinfo==9.0.0
py-trees==0.7.6
pyasn1==0.5.0
pyasn1-modules==0.3.0
pycocotools==2.0.4
pycodestyle==2.9.1
pycollada==0.7.2
pycparser==2.21
pydot==1.4.2
Pygments==2.16.1
pyliblzfse==0.4.1
pymeshlab==2022.2.post4
pyngrok==7.0.0
pyparsing==3.1.1
pyquaternion==0.9.9
pyrealsense2==2.50.0.3812
pyrsistent==0.18.1
PySocks==1.7.1
pytest==7.0.1
pytest-asyncio==0.18.3
python-box==6.1.0
python-dateutil==2.8.2
python-engineio==4.8.0
python-qt-binding==0.4.4
python-socketio==5.10.0
pytz==2022.1
PyWavelets==1.3.0
PyYAML==6.0
pyzmq==23.0.0
qt-dotgraph==0.4.2
qt-gui==0.4.2
qt-gui-py-common==0.4.2
qtconsole==5.3.1
QtPy==2.1.0
qudida==0.0.4
rawpy==0.18.1
requests==2.31.0
requests-oauthlib==1.3.1
requests-toolbelt==1.0.0
retrying==1.3.4
rich==13.6.0
rosbag==1.16.0
rosclean==1.15.8
rosgraph==1.16.0
roslaunch==1.16.0
roslib==1.15.8
roslz4==1.16.0
rosmaster==1.16.0
rosmsg==1.16.0
rosnode==1.16.0
rosparam==1.16.0
rospy==1.16.0
rosservice==1.16.0
rostest==1.16.0
rostopic==1.16.0
rosunit==1.15.8
roswtf==1.16.0
rqt-console==0.4.12
rqt-controller-manager==0.19.6
rqt-ez-publisher==0.6.1
rqt-ground-robot-teleop==1.4.2
rqt-joint-trajectory-controller==0.21.2
rqt-logger-level==0.4.12
rqt-moveit==0.5.11
rqt-reconfigure==0.5.5
rqt-robot-dashboard==0.5.8
rqt-robot-monitor==0.5.15
rqt-runtime-monitor==0.5.10
rqt-tf-tree==0.6.4
rqt_action==0.4.9
rqt_bag==0.5.1
rqt_bag_plugins==0.5.1
rqt_dep==0.4.12
rqt_graph==0.4.14
rqt_gui==0.5.3
rqt_gui_py==0.5.3
rqt_launch==0.4.9
rqt_msg==0.4.10
rqt_nav_view==0.5.7
rqt_plot==0.4.13
rqt_pose_view==0.5.11
rqt_pr2_dashboard==0.4.0
rqt_publisher==0.4.10
rqt_py_common==0.5.3
rqt_py_console==0.4.10
rqt_py_trees==0.4.1
rqt_robot_steering==0.5.12
rqt_service_caller==0.4.10
rqt_shell==0.4.11
rqt_srv==0.4.9
rqt_top==0.4.10
rqt_topic==0.4.13
rqt_web==0.4.10
rsa==4.8
Rtree==1.1.0
scikit-image==0.19.3
scikit-learn==1.1.1
scipy==1.8.1
seaborn==0.12.2
segmentation-models-pytorch==0.3.0
semantic-version==2.10.0
Send2Trash==1.8.0
sensor-msgs==1.13.1
sentry-sdk==1.34.0
setproctitle==1.3.3
Shapely==1.8.2
shtab==1.6.4
simple-websocket==1.0.0
simplejson==3.19.2
six==1.16.0
smclib==1.8.6
smmap==5.0.0
smplx==0.1.28
sniffio==1.2.0
snowballstemmer==2.2.0
sortedcontainers==2.4.0
soupsieve==2.3.2.post1
Sphinx==5.0.2
sphinxcontrib-applehelp==1.0.2
sphinxcontrib-devhelp==1.0.2
sphinxcontrib-htmlhelp==2.0.0
sphinxcontrib-jsmath==1.0.1
sphinxcontrib-qthelp==1.0.3
sphinxcontrib-serializinghtml==1.1.5
splines==0.3.0
stack-data==0.2.0
svg.path==6.3
sympy==1.12
tabulate==0.8.10
tenacity==8.2.3
tensorboard==2.14.0
tensorboard-data-server==0.7.2
tensorboard-plugin-wit==1.8.1
tensorflow==2.9.1
tensorflow-estimator==2.9.0
tensorflow-io-gcs-filesystem==0.26.0
termcolor==1.1.0
terminado==0.15.0
testpath==0.6.0
tf==1.13.2
tf2-geometry-msgs==0.7.7
tf2-py==0.7.7
tf2-ros==0.7.7
tf2-sensor-msgs==0.7.7
threadpoolctl==3.1.0
tifffile==2022.5.4
timm==0.6.7
tinycss2==1.1.1
tinycudann @ git+https://github.com/NVlabs/tiny-cuda-nn/@212104156403bd87616c1a4f73a1c5f2c2e172a9#subdirectory=bindings/torch
toml==0.10.2
tomli==2.0.1
tomlkit==0.11.0
topic-tools==1.16.0
torch==2.0.1+cu118
torch-fidelity==0.3.0
torch-tb-profiler==0.4.0
torchmetrics==1.2.0
torchvision==0.15.2+cu118
tornado==6.3.3
tqdm==4.64.0
traitlets==5.2.1.post0
trimesh==4.0.2
trio==0.21.0
triton==2.0.0
typeguard==4.1.5
typing_extensions==4.8.0
tyro==0.5.12
ultralytics==8.0.146
unique_id==1.0.6
UnrealCV==0.4.0
urllib3==2.0.7
viser==0.1.3
wandb==0.15.12
wcwidth==0.2.5
webencodings==0.5.1
websocket-client==1.3.2
websockets==12.0
Werkzeug==3.0.1
widgetsnbextension==4.0.9
wrapt==1.15.0
wsproto==1.2.0
wurlitzer==3.0.3
xacro==1.14.16
xatlas==0.0.8
xxhash==3.4.1
yacs==0.1.8
yourdfpy==0.0.53
zipp==3.8.0

[brentyi]

Do you mind also sharing your web browser? And could you try upgrading to viser 0.1.9?

[dvdmc]

Same problem in viser 0.1.9. The browser is Chrome Version 119.0.6045.105 (Official Build) (64-bit) running on Ubuntu 22.04. Same problem on Firefox

[dvdmc]

I changed the line mapping = msgpack.unpackb(message) with mapping = msgpack.unpackb(message, encoding="utf8") and the error is not appearing anymore. I can't really see anything in the example but will look into it tomorrow :) Please, let me know if you need any more tests! (and the reason of the problem if you discover it)

Edit for reference: https://github.com/msgpack/msgpack-python/issues/99

[brentyi]

Thanks! Based on the issue you linked, I was able to reproduce your error by downgrading my msgpack to any version lower than 1.0.0. With msgpack >=1.0.0, I'm able to run everything without issues.

Are you 100% sure there's no environment issue resulting in an older version of msgpack on your end?

[dvdmc]

That seems to be the issue... For some reason, my conda env Python is importing the module from the global msgpack. Thanks for your help and sorry for the inconvenience. I will close the issue, but if you have any insight on why this might be happening it would be helpful for me and future visitors.

[brentyi]

Glad you figured that out! My best guess is that it might be related to the apparent presence of ROS on your system, but pretty hard to say anything for sure...

[dvdmc]

I was able to fix the issue. The problem was that msgpack-python, installed in my global Python, was being imported before msgpack in the conda environment. Probably the issue was not the environment overlay but the different package names.