facebookresearch / habitat-sim

A flexible, high-performance 3D simulator for Embodied AI research.
https://aihabitat.org/
MIT License
2.62k stars 420 forks source link

Incorrect loading of Matterport3D annotations #830

Closed senthilps8 closed 4 years ago

senthilps8 commented 4 years ago

🐛 Bug

The Matterport3D region bounding boxes seem to be loaded incorrectly. Specifically, the sizes attribute of the axis aligned bounding boxes are incorrect. I think it is caused by this line: https://github.com/facebookresearch/habitat-sim/blob/master/src/esp/scene/Mp3dSemanticScene.cpp#L111

The region annotation line in Matterport data has the following format: R region_index level_index 0 0 label px py pz xlo ylo zlo xhi yhi zhi height 0 0 0 0 So the xhi yhi zhi values are converted to their absolute values by habitat_sim when it's loaded.

I was able to fix it by just getting rid of the .abs(), but I'm not sure if that's the best way or why the .abs() was there in the first place

To Reproduce

Steps to reproduce the behavior: Here is a minimal script:

from typing import Callable, List, Union
import numpy as np
import habitat_sim
from habitat_sim import bindings as hsim
from habitat_sim.agent.agent import AgentConfiguration

def _config_sim(scene_filepath):
    settings = {
        "scene": scene_filepath,  # Scene path
        "default_agent": 0,
        "sensor_height": 1.5,  # Height of sensors in meters
        "silent": True,
    }
    sim_cfg = hsim.SimulatorConfiguration()
    sim_cfg.gpu_device_id = 0
    sim_cfg.scene.id = settings["scene"]

    # create agent specifications
    agent_cfg = AgentConfiguration()
    return habitat_sim.Configuration(sim_cfg, [agent_cfg])

cfg = _config_sim('/datasets01/mp3d/090920/17DRP5sb8fy/17DRP5sb8fy.glb')
sim = habitat_sim.Simulator(cfg)
print('Region 0 Center: {}'.format(sim.semantic_scene.regions[0].aabb.center))
print('Region 0 Sizes: {}'.format(sim.semantic_scene.regions[0].aabb.sizes))

The output I see:

Region 0 Center: [-2.0816548  1.259409   2.1036716]
Region 0 Sizes: [17.64369    2.5356617  5.3830767]

From the Matterport annotation file 17DRP5sb8fy.house:

R  0 0  0 0  b  -8.84465 1.00624 -0.00842175  -10.9035 0.587867 -0.00842175  -6.74019 4.79521 2.52724  2.53566  0 0 0 0
R  1 0  0 0  a  -6.2772 -0.34982 0.0086211  -8.13822 -2.74781 0.0086211  -4.37099 0.522308 2.19673  2.18811  0 0 0 0
R  2 0  0 0  t  -3.81385 -0.287231 0.0117457  -4.34634 -1.55754 0.0117457  -3.35606 0.507769 2.19985  2.1881  0 0 0 0
R  3 0  0 0  a  3.60977 -0.68145 -0.00638012  2.46613 -1.99366 -0.00638012  4.63827 0.0578053 2.18173  2.18811  0 0 0 0
R  4 0  0 0  v  2.36032 2.46103 -0.000640154  1.02212 0.445752 -0.000640154  4.63471 5.29584 2.58552  2.58616  0 0 0 0
R  5 0  0 0  d  -0.714269 2.6733 -0.00935112  -2.1758 0.523216 -0.00935112  0.794515 5.15928 2.57681  2.58616  0 0 0 0
R  6 0  0 0  e  0.369999 -0.953237 -0.00878062  -0.799924 -2.08913 -0.00878062  2.3481 0.43497 2.27884  2.28762  0 0 0 0
R  7 0  0 0  l  -4.50916 2.59627 0.021234  -6.58635 0.684436 0.021234  -2.12929 5.18929 2.6074  2.58617  0 0 0 0
R  8 0  0 0  f  -9.20954 -0.838467 0.0212027  -11.1912 -2.77622 0.0212027  -8.33375 0.531676 2.60737  2.58617  0 0 0 0
R  9 0  0 0  k  -2.17456 -0.605839 0.00206988  -3.18871 -1.5401 0.00206988  -0.981305 1.15255 2.19018  2.18811  0 0 0 0

Expected behavior

I get this output with the fix I suggested above:

Region 0 Center: [-8.821845   1.259409  -2.6915383]
Region 0 Sizes: [ 4.1633096  2.5356617 -4.207343 ]

System Info

ENVIRONMENT INFO:
Platform: Linux-4.15.0-55-generic-x86_64-with-debian-buster-sid
Machine: x86_64
Processor: x86_64
Libc version: glibc 2.10
Mac version:
Python version: 3.7.7
Architecture: 64bit
Win version:
System OS: Linux
Release: 4.15.0-55-generic
Version: #60-Ubuntu SMP Tue Jul 2 18:22:20 UTC 2019
Operational System: linux
GCC version: b'gcc (Ubuntu 7.4.0-1ubuntu1~18.04.1) 7.4.0\n'
CMAKE version: b'cmake version 3.14.0\n'
NVIDIA-SMI: b'Tue Oct  6 19:40:35 2020       \n+-----------------------------------------------------------------------------+\n| NVIDIA-SMI 418.116.00   Driver Version: 418.116.00   CUDA Version: 10.1     |\n|-------------------------------+----------------------+----------------------+\n| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |\n| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |\n|===============================+======================+======================|\n|   0  Quadro GP100        Off  | 00000000:AF:00.0 Off |                    0 |\n| 27%   43C    P0    31W / 235W |      0MiB / 16278MiB |      0%      Default |\n+-------------------------------+----------------------+----------------------+\n|   1  Quadro GP100        Off  | 00000000:D8:00.0 Off |                    0 |\n| 27%   42C    P0    31W / 235W |      0MiB / 16278MiB |      0%      Default |\n+-------------------------------+----------------------+----------------------+\n'
Pip packages versions:
b''
Conda packages versions:
b'attrs                     20.1.0                     py_0    19.1.0\ngitpython                 3.1.2                    pypi_0    pypi\nimageio                   2.8.0                    pypi_0    pypi\nimageio-ffmpeg            0.4.2                    pypi_0    pypi\nmatplotlib                3.1.3                    py37_0  \nnumba                     0.49.1                   pypi_0    pypi\nnumpy                     1.18.1           py37h4f9e942_0  \nnumpy-quaternion          2020.5.19.15.27.24          pypi_0    pypi\npillow                    7.2.0            py37hb39fc2d_0  \nscipy                     1.4.1                    pypi_0    pypi  1.3.0\ntqdm                      4.48.2                     py_0  \n'
erikwijmans commented 4 years ago

I added that abs to handle negative sizes as those don't make sense. I for some reason thought that the constructor/mp3d where in center/size format not min/max format.

https://github.com/facebookresearch/habitat-sim/pull/832 should fix this. Note that it will produce sizes that are all positive.