Class Layer Missclassification problem

[marcokat81]

Hi @Schmluk ,

I am facing an issue in which for many submap allocations voxels of other semantic classes are grouped into 1 submap. Due to inaccurate allocations the tracker tries to track those falsely classified voxels and integrates them . This is causing the dynamic objects to not getting cleared from the map.

Here is an example of the rendered image in which it allocates and classifies the ground pixel voxels into a single submap. It classifies the static pixels on ground as a human.

I am using moving_binary_count as the classification method and my segmentation image is accurate as well.

Any idea why this issue might happen ? Is there a way to avoid this ?

[marcokat81]

Hi @Schmluk ,

The getBelongingProbability values from the classifier for those miss classified voxels are usually less than 0.5. I wonder why they are integrated into a different submap in this case. I am not how the increment count are happening here?

I changed the belongsToSubmap function to call getBelongingProbability inside the moving_binary_count, then the issue was not this significant. the mesh generated was slightly accurate. The mesh generated by generateClassificationMesh inside the submap_visualizer was even better in comparison. Is generateClassificationMesh doing something different than regular mesh_integrator that the submap interface uses here ?

Is is possible to trim/update the tsdf blocks used for mesh generation based on its class layers blocks belonging probability ? (so that they don't get falsely tracked and integrated). Where can I make such a change ?

[Schmluk]

Hi @marcokat81

The idea is that each submap contains all geometry, and then labels each voxel as belonging to the submap or not.

The line you highlighted states that if the classes of input and submap are existent and identical, the binary count is incremented by 1, else by 0.

The mesher should already only mesh voxels that are labeled as belonging to the submap, as can be seen here. Is classification enabled in the submap mesher config? Yes, generateClassificationMesh does something different, it meshes the entire geometry (not only the belonging part) and colors it according to the belonging probability. But this is happening outside the submap mesh so it should not be used for tracking.

[marcokat81]

@Schmluk ....Yes I have enabled the classification and I saw that the mesh integrator calls the belongsToSubmap.

what determines which classes/ids are initialized into a specific/new allocated submap ? where does this happen ?

I also have a naive question ... how to do you define a geometry over here ? I am asking if I can control the allocation of the geometry for specific submaps. I noticed sometimes that within a submap some blocks that are allocated are pretty far away from its majority allocated blocks (due to erroneous depth). I was wondering if I could filter/remove those blocks which are not part of the connected region within its submap ? Is there a way of doing this ?
Specifically my question is does the submap/mesh generation do a block connected region check within its own submap while trying to determine its geometry ?

[Schmluk]

The ID trackers map input IDs to submap IDs, everything downstream happens on the level of submap IDs.

The geometry is represented by the TSDF blocks/voxels. You can control the allocation of blocks in the TSDF (class) integrator, or remove e.g. empty blocks in the map manager. Currently there is no connectedness check, but that should be straight forward to add to the allocation or management. The easiest solution for your case would probably be to denoise the depth image though?

[Schmluk]

Closed due to inactivity.