[Bug] Sharkbite Bug - all segmentations have a visual bug impacting ink detection training.

[hariseldon137]

What happened?

Looks like a meshing bug maybe - there are visual artefacts throughout every segment we have done so far, which will prevent the correct training of ink detection algorithms. Image posted in the discord.

This has been isolated as an artefact of moving the segmentation line too far on a given segment.

This still needs to be carefully debugged/understood, so we can prevent it from happening.

There is a smoothing function to mitigate this to some extent, but we need to know what it is doing - Laplacian will damage our data.

Steps to reproduce

All segmentations.

Version

all versions

How did you install the software?

both source and docker builds for segmentation, only docker builds for running vc_render

On which operating systems have you experienced this issue?

• [ ] macOS
• [ ] Windows
• [X] Linux

Relevant log output

No response

Code of Conduct

• [X] I agree to follow this project's Code of Conduct

[csparker247]

Please post the image, a screenshot of the same error in 3D, and if possible, a zip containing the textured mesh (obj + tif + mtl).

[hariseldon137]

I have given an example here, where the segmentation line has been pulled too far in VC, leaving a steep edge in the 3D image, and sharkbites in the tif (centered in each frame). I've been running the smoothing function in VC and it gets rid of 3/4 of the sharkbites. RICHI is saying his new algorithm is good enough to not have this happen, and we will be able to inspect his results hopefully in the coming week. So with any luck we have sidestepped the issue here.

12931.zip

[schillij95]

Smoothing the mesh (--mesh-resample-smoothing 3) during vc_render makes this bug less severe. example command:

vc_render -v ../ -s {segmentation number} -o out.obj --output-ppm out.ppm --mesh-resample-smoothing 3
vc_layers_from_ppm -v ../ -p out.ppm --output-dir layers/

This is an easy fix that could also be run on the already existing segments

[csparker247]

Yeah, as shown in your images, the surface "significantly" changed shape between two slices (significantly as far as the meshing is concerned). I don't think this is likely to cause much issue for any downstream tasks like ink-id or anything of that sort.

I'm not quite sure what one would do to fix this in any sort of automatic sense. It's possible that, in general, someone needs to do that with the surface they're segmenting and it just so happens to be wrong here. I don't know. This feels like something a good segmentation editor might be able to handle.