navis-org / skeletor

Extraction of 3D skeletons from meshes.
https://navis-org.github.io/skeletor/
GNU General Public License v3.0
210 stars 26 forks source link

too many "bones" in my skeleton? #48

Open winkdc opened 3 months ago

winkdc commented 3 months ago

The skeletons that I generate seem to have too many edges with overlaps and tiny branches, especially in proximity of intersections. Is there a method I should be using to reduce the skeleton to something more streamlined? pic1 pic2 test.txt stent7_stl.txt

schlegelp commented 3 months ago

Thanks for sharing!

I take it the screenshot is showing the content of what your code saves as vertices_%s.txt and edges_%s.txt?

Assuming that's the case then you are saving the contracted mesh rather than the processed skeleton. The skel.mesh property is the input mesh you passed to the skeletonization function (see the docstring for the Skeleton class), which in your case is the contracted mesh.

What you want to use are the .vertices and .edges properties (or alternatively the .swc table).

Let me know if you need additional pointers (or if I have misinterpreted your code).

schlegelp commented 3 months ago

On a side note: skeletor will produce acyclic tree-like skeletons. That's primarily because I'm working with neurons which we expect to be acyclic, i.e. cycles are to be avoided.

Your mesh will (correctly) result in a skeleton with cycles - which skeletor will subsequently remove assuming they are pathologic. So as things stand you will see breaks in the skeleton. I have meant to make the cycle-breaking optional but haven't gotten around to doing that yet (for some methods it's actually also more complicated as it is baked in).

One work-around would be to use the vertex positions from the skeleton but the connectivity from the original mesh (a mapping between original vertices -> skeleton vertices is stored as Skeleton. mesh_map property).

winkdc commented 3 months ago

Thank you for the reply. Yes, the screenshot is showing the content of what mycode saves as vertices_%s.txt and edges_%s.txt

I cast a vote for adding cycle-breaking as an option. For my geometry, the skeletons I get from skel.mesh seem to have significant potential value - if I could just reduce the many nearly coincident vertices and edges into a smaller set.

I am a bit confused about your suggesed work-around. In my code snippet, are you saying to use skel.mesh.vertices[skel.mesh_map] instead of skel.mesh.vertices?

schlegelp commented 3 months ago

Sorry, I think we have crossed some wires here: skel.mesh is not the skeleton, it's the contracted mesh. I.e. skel.mesh.vertices and skel.mesh.edges is simply the original mesh's vertices and the edges making up faces after the vertices have been pulled together to contract the mesh.

Here is a quick example:

>>> import skeletor as sk 
>>> import trimesh as tm
>>> # Load the mesh 
>>> m = tm.load('stent7.stl')
>>> # Fix potential issues with the mesh
>>> fixed = sk.pre.fix_mesh(m)
>>> # Contract the mesh
>>> # (this is entirely optional and can be finicky but seem to work well for your mesh)
>>> cont = sk.pre.contract(fixed)
>>> # Skeletonize the contracted mesh
>>> skel = sk.skeletonize.by_teasar(cont, inv_dist=1)

The skeleton representation is stored as skel.vertices and skel.edges:

>>> skel.vertices
array([[ 0.92641366, 56.49105805, 24.53067979],
       [ 0.95698625, 55.85853334, 24.18338986],
       [ 0.86943069, 28.64119572, 24.39395668],
       ...,
       [ 1.16120401, 73.83841077, 23.70497722],
       [45.38167596, 27.16261274, 52.18504955],
       [12.67703859, 57.79743991,  5.82538293]])
>>> skel.edges 
array([[   1,    0],
       [   2,  559],
       [   3,  398],
       ...,
       [1369,  752],
       [1370, 1154],
       [1371,  870]])

Visualizing these vertices + edges gets us this:

skeleton

As you can see this now is a proper simplified, skeletal representation of your mesh. There is obviously room for improvement - I didn't bother trying different methods or adjusting parameters.

The other thing you probably noticed in the screenshot is that the skeleton contains breaks while the mesh is obviously contiguous. That's the aforementioned cycle breaking that's currently baked into skeletor.

There are two potential ways of "mending" these breaks:

  1. Do not introduce breaks the skeleton in the first place (might be difficult as it's inherent to some of the methods)
  2. Heal breaks afterwards by using the original mesh's connectivity

I had a quick crack at both these options and neither is trivial. Will have to look into it.

schlegelp commented 3 months ago

I just added a new method to the Skeleton class: Skeleton.mend_breaks() will (re-)introduce edges that are present in the mesh but not the skeleton. This works by comparing the connectivity of the original mesh with that of the skeleton. If the shortest path between two adjacent vertices on the mesh is shorter than the distance between the nodes in the skeleton, a new edge is added to the skeleton.

If you re-install skeletor from Github you can try it out:

>>> # Starting from example above
>>> edges, vertices = skel.mend_breaks()

Screenshot 2024-07-23 at 10 58 26

There is a bit of a trade off as we introduce both true positive as well as false positive new edges but it doesn't look too bad. Some of remaining flaws like the bits sticking out (noise from the contraction) could be removed through further post-processing.

winkdc commented 3 months ago

Thank you for the clarification and for helping to fill the skeleton gaps! Any further enhancements to counter the intended cycle breaking would be welcome. For now, manually adding new bones to the skeleton seems like the way to go.