search

MIT-SPARK / Kimera-Semantics

Real-Time 3D Semantic Reconstruction from 2D data
BSD 2-Clause "Simplified" License
631 stars 134 forks source link

Semantic Point Cloud's input for inserting into mesh #54

Closed codieboomboom closed 3 years ago

codieboomboom commented 3 years ago

Hi can I check that for the Kimera Semantics, we just need Depth (from camera) and 2D semantic image and pose from some SLAM backend?

The input semantic image is the actual colored image and not the label matrix? And similarly there is no need to provide the confidence matrix for the semantic image?

We are assuming that the semantic module is very robust in such case?

Thank you

RozDavid commented 3 years ago

Hey @AnhTuDo1998,

I am neither a pro or the maintainer of this project, but let me share what I understood:

Here your input semantic image should contain the only the highest probability label, more precisely the color represented by the index of that label. In runtime the algorithm translated the color code to the label index, accepts every measurement as if that would be the ground truth label and aggregates label predicitons for every individual voxel with a measurement frequency. The corresponding snippet can be found here.

With this a label confidence matrix can be calculated and iteratively updated with latest measurements integrated into the measurement frequency vector and multiplied with the previous likelihood.

Hope this could help and I was right with my understanding. Bests, David

codieboomboom commented 3 years ago

Thanks @RozDavid just what I am looking for. Closing thread since question answered