NVIDIA-ISAAC-ROS / isaac_ros_pose_estimation

Deep learned, NVIDIA-accelerated 3D object pose estimation
https://developer.nvidia.com/isaac-ros-gems
Apache License 2.0
142 stars 21 forks source link

Regarding Object scale for the centerpose training data #30

Closed ArghyaChatterjee closed 10 months ago

ArghyaChatterjee commented 10 months ago

Hello,

I have followed your tutorial of getting the data from objectron website to export as frames and jsons. Now one thing to notice is that I am getting object scale as:

            "scale": [
                0.14000000059604645,
                0.10999999940395355,
                0.09000000357627869

Here is the picture:

00435

Here is the corresponding json file:

{
    "AR_data": {
        "plane_center": [
            0.04741383343935013,
            -0.043089061975479126,
            -0.3472689986228943
        ],
        "plane_normal": [
            -0.7107083201408386,
            -0.030043570324778557,
            0.7028449773788452
        ]
    },
    "camera_data": {
        "camera_projection_matrix": [
            [
                1.5488815307617188,
                0.0,
                0.01502901315689087,
                0.0
            ],
            [
                0.0,
                2.0651752948760986,
                0.008649170398712158,
                0.0
            ],
            [
                0.0,
                0.0,
                -0.9999997615814209,
                -0.0009999998146668077
            ],
            [
                0.0,
                0.0,
                -1.0,
                0.0
            ]
        ],
        "camera_view_matrix": [
            [
                0.3634883463382721,
                -0.7107083201408386,
                0.6023037433624268,
                -0.0781826600432396
            ],
            [
                0.8397062420845032,
                -0.03004208207130432,
                -0.5422091484069824,
                -0.3924746811389923
            ],
            [
                0.4034470021724701,
                0.7028449773788452,
                0.5858662724494934,
                -0.10444661229848862
            ],
            [
                0.0,
                0.0,
                0.0,
                1.0
            ]
        ],
        "height": 800,
        "intrinsics": {
            "cx": 302.3864237467448,
            "cy": 393.7800598144531,
            "fx": 619.5526123046875,
            "fy": 619.5526123046875
        },
        "location_world": [
            0.4001206159591675,
            0.0060539511032402515,
            -0.10452190786600113
        ],
        "quaternion_world_xyzw": [
            -0.44935069040050507,
            -0.07176909303646607,
            -0.5595578534243499,
            0.6926962992484125
        ],
        "width": 600
    },
    "objects": [
        {
            "class": "cup",
            "keypoints_3d": [
                [
                    0.008324876427650452,
                    -0.04474139213562012,
                    -0.30861252546310425
                ],
                [
                    0.09943512827157974,
                    -0.0072012245655059814,
                    -0.2931317090988159
                ],
                [
                    0.04458438605070114,
                    0.04007652401924133,
                    -0.3465753197669983
                ],
                [
                    0.021257229149341583,
                    -0.010505825281143188,
                    -0.21581879258155823
                ],
                [
                    -0.03359351307153702,
                    0.03677186369895935,
                    -0.2692623734474182
                ],
                [
                    0.05024326592683792,
                    -0.12625464797019958,
                    -0.34796270728111267
                ],
                [
                    -0.004607461392879486,
                    -0.07897695899009705,
                    -0.40140628814697266
                ],
                [
                    -0.027934633195400238,
                    -0.12955927848815918,
                    -0.2706497609615326
                ],
                [
                    -0.08278535306453705,
                    -0.08228158950805664,
                    -0.3240933418273926
                ]
            ],
            "location": [
                0.008324880655670608,
                -0.04474139541170574,
                -0.30861252277641116
            ],
            "mug": true,
            "mug_handle_visible": true,
            "name": "cup_0",
            "projected_cuboid": [
                [
                    207,
                    410
                ],
                [
                    282,
                    604
                ],
                [
                    369,
                    473
                ],
                [
                    267,
                    455
                ],
                [
                    382,
                    316
                ],
                [
                    71,
                    483
                ],
                [
                    175,
                    386
                ],
                [
                    0,
                    329
                ],
                [
                    139,
                    235
                ]
            ],
            "provenance": "objectron",
            "quaternion_xyzw": [
                0.5640233208476352,
                0.6919439846217341,
                -0.4437327221383519,
                -0.07869235661416642
            ],
            "scale": [
                0.14000000059604645,
                0.10999999940395355,
                0.09000000357627869
            ]
        }
    ]
}

Now, the problem is in which unit does this scale is ??

The actual scale looks like this (if it's in meters):

                0.014000000059604645,
                0.010999999940395355,
                0.009000000357627869

Why divided by 10 ? (nor does it fit in meters. not even in centemeters)

ArghyaChatterjee commented 10 months ago

Sorry, looks like there is a mistake from my side regarding the cuboid dimension calculation. Closing the issue.

monajalal commented 6 months ago

@ArghyaChatterjee how did you produce AR_data for your custom data in Isaac Sim replicator?