niessner / Matterport

Matterport3D is a pretty awesome dataset for RGB-D machine learning tasks :)
https://niessner.github.io/Matterport/
MIT License
945 stars 154 forks source link

How to sort panorama_uuid(tripod placement) under a scan in proper order to run a SLAM? #56

Open LongruiDong opened 2 years ago

LongruiDong commented 2 years ago

Hi:

Thanks for contributing such an amazing dataset! I want to test a large scale rgb-d slam method, this data indeed covers large enough indoor scene . But I can't directly get the time order of data collection from [panorama_uuid], if I input rgb-d in the default order, obviously there is almost no overlapping area of the images collected from different tripod placements, which apparently makes it impossible to run a SLAM algorithm.

so how to sort these panorama_uuid to get a normal rgb-d data sequence?

Thanks in advance!

LongruiDong commented 2 years ago

Sorry, Maybe I didn't express it correctly before. I hope I can make it clear this time, thank you for your patience again!

For SLAM, there are often overlapping areas between adjacent frames of the input image.

If we just randomly input all 18 images([1]_[0-2]_[0-5]) under different [panorama_uuid], we may meet this situation: image

Above figure is from the dataset paper, each green dot represent a location, which is denoted by a [panorama_uuid]. I know there are 18 images under a [panorama_uuid].

I use red number : '1','2'to name two [panorama_uuid] as example.

At first all images at [1] are input, then maybe [2]_[0-2]_[0-5].jpg are input to SLAM.

however, there is no visual overlap between the last image in [1]_[0-2]_[0-5] and first image in [2]_[0-2]_[0-5], which will cause the algorithm to lose track...

Ideally, we should input the images to SLAM in the order of the blue arrows(there may be other orders, as long as there are reasonable overlaps).

So I would like to ask, is there a way to easily get a reasonable order of [panorama_uuids] in one scene from Matterport3d

OR get a set of rgb-d data and the corresponding GT 6dof poses from Matterport3d to train a SLAM?

guangkaixu commented 1 year ago

Sorry, Maybe I didn't express it correctly before. I hope I can make it clear this time, thank you for your patience again!

For SLAM, there are often overlapping areas between adjacent frames of the input image.

If we just randomly input all 18 images([1]_[0-2]_[0-5]) under different [panorama_uuid], we may meet this situation: image

Above figure is from the dataset paper, each green dot represent a location, which is denoted by a [panorama_uuid]. I know there are 18 images under a [panorama_uuid].

I use red number : '1','2'to name two [panorama_uuid] as example.

At first all images at [1] are input, then maybe [2]_[0-2]_[0-5].jpg are input to SLAM.

however, there is no visual overlap between the last image in [1]_[0-2]_[0-5] and first image in [2]_[0-2]_[0-5], which will cause the algorithm to lose track...

Ideally, we should input the images to SLAM in the order of the blue arrows(there may be other orders, as long as there are reasonable overlaps).

So I would like to ask, is there a way to easily get a reasonable order of [panorama_uuids] in one scene from Matterport3d

OR get a set of rgb-d data and the corresponding GT 6dof poses from Matterport3d to train a SLAM?

Hi, have you found any possibility to train a SLAM with the Matterport3d ?

TRINHHOANGANH commented 1 year ago

@LongruiDong How to add coordinates automatically between keyframes in SLAM. image