Simulation loading indefinitely in Jupyter Notebook

pfmephisto commented 1 year ago

habitat-sim == 0.2.4 installed through conda

I am experiencing a bug when trying to initialise a simulation that contains sematic data in a Jupyter Notebook. jupyterhub==2.0.0 I'm using the HM3D dataset and the issue is that the Jupyter cell will keep running indefinitely, without returning any output or error messages. The issue seems to appear when the scene_dataset_config_file is being supplied. If omit or a wrong file is used the simulation loads fine, just without the sematic mesh.


import os 
import habitat_sim
import matplotlib.pyplot as plt

sim_settings = {
     "scene": '/hm3d_fullpath/minival/00808-y9hTuugGdiq/y9hTuugGdiq.basis.glb', 
    "scene_dataset": '/hm3d_fullpath/hm3d_annotated_basis.scene_dataset_config.json',
    "default_agent": 0,
    "sensor_height": 1.5,  # Height of sensors in meters
    "width": 512,  # Spatial resolution of the observations
    "height": 512,
    "hfov": 90,
}

def make_simple_cfg(settings):    
    # simulator backend
    sim_cfg = habitat_sim.SimulatorConfiguration()
    sim_cfg.gpu_device_id = 0
    sim_cfg.create_renderer = True
    sim_cfg.enable_physics = False
    sim_cfg.scene_id = settings["scene"]
    sim_cfg.scene_dataset_config_file = settings["scene_dataset"]
    sim_cfg.load_semantic_mesh = True
    sim_cfg.requires_textures = True

    sensor_position = [0.0, settings["sensor_height"], 0.0]
    sensor_resolution = [settings["height"], settings["width"]]

    sensor_specifications = []
    for sensor_name, sensor_type in zip(
        [ "color_sensor", "depth_sensor", "semantic_sensor" ],
        [ habitat_sim.SensorType.COLOR, habitat_sim.SensorType.DEPTH, habitat_sim.SensorType.SEMANTIC ]
    ):
        sensor_spec  = habitat_sim.CameraSensorSpec()
        sensor_spec.uuid = sensor_name
        sensor_spec.hfov = settings["hfov"]
        sensor_spec.sensor_type = sensor_type
        sensor_spec.resolution = sensor_resolution
        sensor_spec.position = sensor_position
        sensor_spec.sensor_subtype = habitat_sim.SensorSubType.PINHOLE
        sensor_spec.noise_model = "None"

        sensor_specifications.append(sensor_spec)

    # agent
    agent_cfg = habitat_sim.agent.AgentConfiguration()
    agent_cfg.sensor_specifications = sensor_specifications

    rotation = 60.0
    agent_cfg.action_space = {
        "move_forward": habitat_sim.agent.ActionSpec(
            "move_forward", habitat_sim.agent.ActuationSpec(amount=0.25)
        ),
        "turn_left": habitat_sim.agent.ActionSpec(
            "turn_left", habitat_sim.agent.ActuationSpec(amount=rotation)
        ),
        "turn_right": habitat_sim.agent.ActionSpec(
            "turn_right", habitat_sim.agent.ActuationSpec(amount=rotation)
        ),
    }

    return habitat_sim.Configuration(sim_cfg, [agent_cfg])

cfg = make_simple_cfg(sim_settings)
sim = habitat_sim.Simulator(cfg)  #<-- This does not seem to return

Expected behavior

The notebook cell finishes running and returns a simulation object.

Additional context

The issue seems only to appear in a notebook. If the same code is run in a regular python file in the same conda environment it is working and the sematic data is loaded just fine.

System Info

ENVIRONMENT INFO: Platform: Linux-5.19.0-41-generic-x86_64-with-glibc2.35 Machine: x86_64 Processor: x86_64 Libc version: glibc 2.35 Mac version: Python version: 3.9.16 Architecture: 64bit ELF Win version: System OS: Linux Release: 5.19.0-41-generic Version: #42~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Tue Apr 18 17:40:00 UTC 2 Operational System: linux GCC version: 'gcc (Ubuntu 11.3.0-1ubuntu1~22.04) 11.3.0' CMAKE version: 'cmake version 3.14.0' "NVIDIA-SMI: +-----------------------------------------------------------------------------+ | NVIDIA-SMI 525.105.17 Driver Version: 525.105.17 CUDA Version: 12.0 | |-------------------------------+----------------------+----------------------+ | GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. | | | | MIG M. | |===============================+======================+======================| | 0 NVIDIA RTX A6000 Off | 00000000:C1:00.0 Off | Off | | 30% 38C P8 28W / 300W | 2964MiB / 49140MiB | 0% Default | | | | N/A | +-------------------------------+----------------------+----------------------+ Pip packages versions: asttokens 2.2.1 attrs 23.1.0 backcall 0.2.0 backports.functools-lru-cache 1.6.4 brotlipy 0.7.0 certifi 2023.5.7 cffi 1.15.1 charset-normalizer 3.1.0 colorama 0.4.6 comm 0.1.3 contourpy 1.0.7 cryptography 40.0.2 cycler 0.11.0 dataclasses 0.8 debugpy 1.6.7 decorator 5.1.1 executing 1.2.0 fonttools 4.39.3 freetype-py 2.4.0 future 0.18.3 gitdb 4.0.10 GitPython 3.1.31 habitat-sim 0.2.4 idna 3.4 imageio 2.28.1 imageio-ffmpeg 0.4.8 importlib-metadata 6.6.0 importlib-resources 5.12.0 ipykernel 6.23.0 ipython 8.13.2 ipywidgets 8.0.6 jedi 0.18.2 jupyter_client 8.2.0 jupyter_core 5.3.0 jupyterlab-widgets 3.0.7 kiwisolver 1.4.4 llvmlite 0.39.1 magnum 0.0.0 matplotlib 3.7.1 matplotlib-inline 0.1.6 munkres 1.1.4 nest-asyncio 1.5.6 networkx 3.1 numba 0.56.4 numpy 1.23.5 numpy-quaternion 2022.4.3 packaging 23.1 parso 0.8.3 pexpect 4.8.0 pickleshare 0.7.5 Pillow 9.5.0 pip 23.1.2 platformdirs 3.5.0 ply 3.11 pooch 1.7.0 prompt-toolkit 3.0.38 psutil 5.9.5 ptyprocess 0.7.0 pure-eval 0.2.2 pycparser 2.21 pyglet 1.5.27 Pygments 2.15.1 PyOpenGL 3.1.6 pyOpenSSL 23.1.1 pyparsing 3.0.9 PyQt5 5.15.7 PyQt5-sip 12.11.0 pyrender 0.1.45 PySocks 1.7.1 python-dateutil 2.8.2 pyzmq 25.0.2 requests 2.29.0 scipy 1.10.1 setuptools 67.7.2 sip 6.7.9 six 1.16.0 smmap 3.0.5 stack-data 0.6.2 toml 0.10.2 tomli 2.0.1 tornado 6.3 tqdm 4.65.0 traitlets 5.9.0 trimesh 3.21.5 typing_extensions 4.5.0 unicodedata2 15.0.0 urllib3 1.26.15 wcwidth 0.2.6 wheel 0.40.0 widgetsnbextension 4.0.7 xeus-python-shell 0.5.0 xvfbwrapper 0.2.9 zipp 3.15.0

Conda packages versions: _libgcc_mutex 0.1 conda_forge conda-forge _openmp_mutex 4.5 2_gnu conda-forge alsa-lib 1.2.8 h166bdaf_0 conda-forge aom 3.5.0 h27087fc_0 conda-forge asttokens 2.2.1 pyhd8ed1ab_0 conda-forge attr 2.5.1 h166bdaf_1 conda-forge attrs 23.1.0 pyh71513ae_1 conda-forge backcall 0.2.0 pyh9f0ad1d_0 conda-forge backports 1.0 pyhd8ed1ab_3 conda-forge backports.functools_lru_cache 1.6.4 pyhd8ed1ab_0 conda-forge brotli 1.0.9 h166bdaf_8 conda-forge brotli-bin 1.0.9 h166bdaf_8 conda-forge brotlipy 0.7.0 py39hb9d737c_1005 conda-forge bzip2 1.0.8 h7f98852_4 conda-forge c-ares 1.18.1 h7f98852_0 conda-forge ca-certificates 2023.5.7 hbcca054_0 conda-forge cairo 1.16.0 h35add3b_1015 conda-forge certifi 2023.5.7 pyhd8ed1ab_0 conda-forge cffi 1.15.1 py39he91dace_3 conda-forge charset-normalizer 3.1.0 pyhd8ed1ab_0 conda-forge cmake 3.14.0 hf94ab9c_0 conda-forge colorama 0.4.6 pyhd8ed1ab_0 conda-forge comm 0.1.3 pyhd8ed1ab_0 conda-forge contourpy 1.0.7 py39h4b4f3f3_0 conda-forge cryptography 40.0.2 py39h079d5ae_0 conda-forge curl 8.0.1 h588be90_0 conda-forge cycler 0.11.0 pyhd8ed1ab_0 conda-forge dataclasses 0.8 pyhc8e2a94_3 conda-forge dbus 1.13.6 h5008d03_3 conda-forge debugpy 1.6.7 py39h227be39_0 conda-forge decorator 5.1.1 pyhd8ed1ab_0 conda-forge executing 1.2.0 pyhd8ed1ab_0 conda-forge expat 2.5.0 hcb278e6_1 conda-forge ffmpeg 5.1.2 gpl_h8dda1f0_106 conda-forge font-ttf-dejavu-sans-mono 2.37 hab24e00_0 conda-forge font-ttf-inconsolata 3.000 h77eed37_0 conda-forge font-ttf-source-code-pro 2.038 h77eed37_0 conda-forge font-ttf-ubuntu 0.83 hab24e00_0 conda-forge fontconfig 2.14.2 h14ed4e7_0 conda-forge fonts-conda-ecosystem 1 0 conda-forge fonts-conda-forge 1 0 conda-forge fonttools 4.39.3 py39h72bdee0_0 conda-forge freetype 2.12.1 hca18f0e_1 conda-forge freetype-py 2.4.0 pyhd8ed1ab_0 conda-forge future 0.18.3 pyhd8ed1ab_0 conda-forge gettext 0.21.1 h27087fc_0 conda-forge gitdb 4.0.10 pyhd8ed1ab_0 conda-forge gitpython 3.1.31 pyhd8ed1ab_0 conda-forge glib 2.76.2 hfc55251_0 conda-forge glib-tools 2.76.2 hfc55251_0 conda-forge gmp 6.2.1 h58526e2_0 conda-forge gnutls 3.7.8 hf3e180e_0 conda-forge graphite2 1.3.13 h58526e2_1001 conda-forge gst-plugins-base 1.22.0 h4243ec0_2 conda-forge gstreamer 1.22.0 h25f0c4b_2 conda-forge habitat-sim 0.2.4 pypi_0 pypi habitat-sim-mutex 1.0 display_bullet aihabitat harfbuzz 6.0.0 h3ff4399_1 conda-forge icu 72.1 hcb278e6_0 conda-forge idna 3.4 pyhd8ed1ab_0 conda-forge imageio 2.28.1 pyh24c5eb1_0 conda-forge imageio-ffmpeg 0.4.8 pyhd8ed1ab_0 conda-forge importlib-metadata 6.6.0 pyha770c72_0 conda-forge importlib-resources 5.12.0 pyhd8ed1ab_0 conda-forge importlib_metadata 6.6.0 hd8ed1ab_0 conda-forge importlib_resources 5.12.0 pyhd8ed1ab_0 conda-forge ipykernel 6.23.0 pyh210e3f2_0 conda-forge ipython 8.13.2 pyh41d4057_0 conda-forge ipywidgets 8.0.6 pyhd8ed1ab_0 conda-forge jedi 0.18.2 pyhd8ed1ab_0 conda-forge jupyter_client 8.2.0 pyhd8ed1ab_0 conda-forge jupyter_core 5.3.0 py39hf3d152e_0 conda-forge jupyterlab_widgets 3.0.7 pyhd8ed1ab_0 conda-forge keyutils 1.6.1 h166bdaf_0 conda-forge kiwisolver 1.4.4 py39hf939315_1 conda-forge krb5 1.20.1 h81ceb04_0 conda-forge lame 3.100 h166bdaf_1003 conda-forge lcms2 2.15 haa2dc70_1 conda-forge ld_impl_linux-64 2.40 h41732ed_0 conda-forge lerc 4.0.0 h27087fc_0 conda-forge libblas 3.9.0 16_linux64_openblas conda-forge libbrotlicommon 1.0.9 h166bdaf_8 conda-forge libbrotlidec 1.0.9 h166bdaf_8 conda-forge libbrotlienc 1.0.9 h166bdaf_8 conda-forge libcap 2.67 he9d0100_0 conda-forge libcblas 3.9.0 16_linux64_openblas conda-forge libclang 16.0.3 default_h83cc7fd_0 conda-forge libclang13 16.0.3 default_hd781213_0 conda-forge libcups 2.3.3 h36d4200_3 conda-forge libcurl 8.0.1 h588be90_0 conda-forge libdeflate 1.18 h0b41bf4_0 conda-forge libdrm 2.4.114 h166bdaf_0 conda-forge libedit 3.1.20191231 he28a2e2_2 conda-forge libev 4.33 h516909a_1 conda-forge libevent 2.1.10 h28343ad_4 conda-forge libexpat 2.5.0 hcb278e6_1 conda-forge libffi 3.4.2 h7f98852_5 conda-forge libflac 1.4.2 h27087fc_0 conda-forge libgcc-ng 12.2.0 h65d4601_19 conda-forge libgcrypt 1.10.1 h166bdaf_0 conda-forge libgfortran-ng 12.2.0 h69a702a_19 conda-forge libgfortran5 12.2.0 h337968e_19 conda-forge libglib 2.76.2 hebfc3b9_0 conda-forge libgomp 12.2.0 h65d4601_19 conda-forge libgpg-error 1.46 h620e276_0 conda-forge libiconv 1.17 h166bdaf_0 conda-forge libidn2 2.3.4 h166bdaf_0 conda-forge libjpeg-turbo 2.1.5.1 h0b41bf4_0 conda-forge liblapack 3.9.0 16_linux64_openblas conda-forge libllvm11 11.1.0 he0ac6c6_5 conda-forge libllvm16 16.0.3 hbf9e925_1 conda-forge libnghttp2 1.52.0 h61bc06f_0 conda-forge libnsl 2.0.0 h7f98852_0 conda-forge libogg 1.3.4 h7f98852_1 conda-forge libopenblas 0.3.21 pthreads_h78a6416_3 conda-forge libopus 1.3.1 h7f98852_1 conda-forge libpciaccess 0.17 h166bdaf_0 conda-forge libpng 1.6.39 h753d276_0 conda-forge libpq 15.2 hb675445_0 conda-forge libsndfile 1.2.0 hb75c966_0 conda-forge libsodium 1.0.18 h36c2ea0_1 conda-forge libsqlite 3.41.2 h2797004_1 conda-forge libssh2 1.10.0 hf14f497_3 conda-forge libstdcxx-ng 12.2.0 h46fd767_19 conda-forge libsystemd0 253 h8c4010b_1 conda-forge libtasn1 4.19.0 h166bdaf_0 conda-forge libtiff 4.5.0 ha587672_6 conda-forge libunistring 0.9.10 h7f98852_0 conda-forge libuuid 2.38.1 h0b41bf4_0 conda-forge libuv 1.44.2 h166bdaf_0 conda-forge libva 2.18.0 h0b41bf4_0 conda-forge libvorbis 1.3.7 h9c3ff4c_0 conda-forge libvpx 1.11.0 h9c3ff4c_3 conda-forge libwebp-base 1.3.0 h0b41bf4_0 conda-forge libxcb 1.13 h7f98852_1004 conda-forge libxkbcommon 1.5.0 h79f4944_1 conda-forge libxml2 2.10.4 hfdac1af_0 conda-forge libzlib 1.2.13 h166bdaf_4 conda-forge llvmlite 0.39.1 py39h7d9a04d_1 conda-forge lz4-c 1.9.4 hcb278e6_0 conda-forge matplotlib 3.7.1 py39hf3d152e_0 conda-forge matplotlib-base 3.7.1 py39he190548_0 conda-forge matplotlib-inline 0.1.6 pyhd8ed1ab_0 conda-forge mpg123 1.31.3 hcb278e6_0 conda-forge munkres 1.1.4 pyh9f0ad1d_0 conda-forge mysql-common 8.0.32 hf1915f5_2 conda-forge mysql-libs 8.0.32 hca2cd23_2 conda-forge ncurses 6.3 h27087fc_1 conda-forge nest-asyncio 1.5.6 pyhd8ed1ab_0 conda-forge nettle 3.8.1 hc379101_1 conda-forge networkx 3.1 pyhd8ed1ab_0 conda-forge nspr 4.35 h27087fc_0 conda-forge nss 3.89 he45b914_0 conda-forge numba 0.56.4 py39h71a7301_1 conda-forge numpy 1.23.5 py39h3d75532_0 conda-forge openh264 2.3.1 hcb278e6_2 conda-forge openjpeg 2.5.0 hfec8fc6_2 conda-forge openssl 3.1.0 hd590300_3 conda-forge p11-kit 0.24.1 hc5aa10d_0 conda-forge packaging 23.1 pyhd8ed1ab_0 conda-forge parso 0.8.3 pyhd8ed1ab_0 conda-forge pcre2 10.40 hc3806b6_0 conda-forge pexpect 4.8.0 pyh1a96a4e_2 conda-forge pickleshare 0.7.5 py_1003 conda-forge pillow 9.5.0 py39h7207d5c_0 conda-forge pip 23.1.2 pyhd8ed1ab_0 conda-forge pixman 0.40.0 h36c2ea0_0 conda-forge platformdirs 3.5.0 pyhd8ed1ab_0 conda-forge ply 3.11 py_1 conda-forge pooch 1.7.0 pyha770c72_3 conda-forge prompt-toolkit 3.0.38 pyha770c72_0 conda-forge prompt_toolkit 3.0.38 hd8ed1ab_0 conda-forge psutil 5.9.5 py39h72bdee0_0 conda-forge pthread-stubs 0.4 h36c2ea0_1001 conda-forge ptyprocess 0.7.0 pyhd3deb0d_0 conda-forge pulseaudio-client 16.1 h5195f5e_3 conda-forge pure_eval 0.2.2 pyhd8ed1ab_0 conda-forge pycparser 2.21 pyhd8ed1ab_0 conda-forge pyglet 1.5.27 py39hf3d152e_3 conda-forge pygments 2.15.1 pyhd8ed1ab_0 conda-forge pyopengl 3.1.6 pyhd8ed1ab_1 conda-forge pyopenssl 23.1.1 pyhd8ed1ab_0 conda-forge pyparsing 3.0.9 pyhd8ed1ab_0 conda-forge pyqt 5.15.7 py39h5c7b992_3 conda-forge pyqt5-sip 12.11.0 py39h227be39_3 conda-forge pyrender 0.1.45 pyh8a188c0_3 conda-forge pysocks 1.7.1 pyha2e5f31_6 conda-forge python 3.9.16 h2782a2a_0_cpython conda-forge python-dateutil 2.8.2 pyhd8ed1ab_0 conda-forge python_abi 3.9 3_cp39 conda-forge pyzmq 25.0.2 py39h0be026e_0 conda-forge qt-main 5.15.8 h5c52f38_10 conda-forge quaternion 2022.4.3 py39h389d5f1_0 conda-forge readline 8.2 h8228510_1 conda-forge requests 2.29.0 pyhd8ed1ab_0 conda-forge rhash 1.4.3 h166bdaf_0 conda-forge scipy 1.10.1 py39he83f1e1_1 conda-forge setuptools 67.6.1 pypi_0 pypi sip 6.7.9 py39h3d6467e_0 conda-forge six 1.16.0 pyh6c4a22f_0 conda-forge smmap 3.0.5 pyh44b312d_0 conda-forge stack_data 0.6.2 pyhd8ed1ab_0 conda-forge svt-av1 1.4.1 hcb278e6_0 conda-forge tk 8.6.12 h27826a3_0 conda-forge toml 0.10.2 pyhd8ed1ab_0 conda-forge tomli 2.0.1 pyhd8ed1ab_0 conda-forge tornado 6.3 py39h72bdee0_0 conda-forge tqdm 4.65.0 pyhd8ed1ab_1 conda-forge traitlets 5.9.0 pyhd8ed1ab_0 conda-forge trimesh 3.21.5 pyhd8ed1ab_0 conda-forge typing-extensions 4.5.0 hd8ed1ab_0 conda-forge typing_extensions 4.5.0 pyha770c72_0 conda-forge tzdata 2023c h71feb2d_0 conda-forge unicodedata2 15.0.0 py39hb9d737c_0 conda-forge urllib3 1.26.15 pyhd8ed1ab_0 conda-forge wcwidth 0.2.6 pyhd8ed1ab_0 conda-forge wheel 0.40.0 pyhd8ed1ab_0 conda-forge widgetsnbextension 4.0.7 pyhd8ed1ab_0 conda-forge withbullet 2.0 0 aihabitat x264 1!164.3095 h166bdaf_2 conda-forge x265 3.5 h924138e_3 conda-forge xcb-util 0.4.0 h166bdaf_0 conda-forge xcb-util-image 0.4.0 h166bdaf_0 conda-forge xcb-util-keysyms 0.4.0 h166bdaf_0 conda-forge xcb-util-renderutil 0.3.9 h166bdaf_0 conda-forge xcb-util-wm 0.4.1 h166bdaf_0 conda-forge xeus 3.0.5 hac2b420_1 conda-forge xeus-python 0.15.9 py39h7633fee_0 conda-forge xeus-python-shell 0.5.0 pyhd8ed1ab_0 conda-forge xeus-python-shell-raw 0.5.0 pyhd8ed1ab_0 conda-forge xeus-zmq 1.0.3 h0541b36_0 conda-forge xkeyboard-config 2.38 h0b41bf4_0 conda-forge xorg-fixesproto 5.0 h7f98852_1002 conda-forge xorg-inputproto 2.3.2 h7f98852_1002 conda-forge xorg-kbproto 1.0.7 h7f98852_1002 conda-forge xorg-libice 1.0.10 h7f98852_0 conda-forge xorg-libsm 1.2.3 hd9c2040_1000 conda-forge xorg-libx11 1.8.4 h0b41bf4_0 conda-forge xorg-libxau 1.0.9 h7f98852_0 conda-forge xorg-libxcursor 1.2.0 h0b41bf4_1 conda-forge xorg-libxdmcp 1.1.3 h7f98852_0 conda-forge xorg-libxext 1.3.4 h0b41bf4_2 conda-forge xorg-libxfixes 5.0.3 h7f98852_1004 conda-forge xorg-libxi 1.7.10 h7f98852_0 conda-forge xorg-libxinerama 1.1.5 h27087fc_0 conda-forge xorg-libxrandr 1.5.2 h7f98852_1 conda-forge xorg-libxrender 0.9.10 h7f98852_1003 conda-forge xorg-randrproto 1.5.0 h7f98852_1001 conda-forge xorg-renderproto 0.11.1 h7f98852_1002 conda-forge xorg-xextproto 7.3.0 h0b41bf4_1003 conda-forge xorg-xf86vidmodeproto 2.3.1 h7f98852_1002 conda-forge xorg-xproto 7.0.31 h7f98852_1007 conda-forge xvfbwrapper 0.2.9 pyhd8ed1ab_1005 conda-forge xz 5.2.6 h166bdaf_0 conda-forge zeromq 4.3.4 h9c3ff4c_1 conda-forge zipp 3.15.0 pyhd8ed1ab_0 conda-forge zlib 1.2.13 h166bdaf_4 conda-forge zstd 1.5.2 h3eb15da_6 conda-forge

frank-gallagher commented 8 months ago

I am experiencing the same issue with habitat-sim=0.3.0 installed with conda.

Did you ever find a solution?

pfmephisto commented 8 months ago

It's a while back and I haven't been working with it since. But if I recall correctly I gave up on the Jupyter notebook and used a standalone python script.

facebookresearch / habitat-sim