Add operator that automatically generates mesh from image's alpha

[Aodaruma]

Is your feature request related to a problem? Please describe. In standard 2D cutout animation applications like Live2D and Spine, the ability to automatically generate meshes from image transparency is essential. Creating meshes manually for a large number of splines is a labor-intensive task and it hinders the creation of detailed and smooth animations. To address this issue, COA Tools 2 should include an automatic mesh generation feature.

Describe the solution you'd like Automatically generating meshes from image transparency is not an easy task. I have referred to several Python libraries and GitHub repositories, but many of them utilize libraries not built into Blender (such as OpenCV, OpenMesh, MeshLib, etc.). However, if it can be easily achieved by incorporating these libraries, I am willing to introduce them.

Describe alternatives you've considered On the other hand, I have discovered that Blender has an operator called "trace_image" in Grease Pencil. Although it is unclear whether it can be intentionally used for automatic mesh generation, if we stick to using the features built into Blender, this seems to be the most viable option.

Additional context If there are any other good approaches, I would appreciate discussing them together.

[Aodaruma]

I'm making operators on feature-automesh-#6

[tozpeak]

Here is my opinion from tech artist perspective.

The automesh system could be cool is theory, but it has both challenges to implement and mediocre result even in proprietary software:

• From tech perspective, it is needed to keep count of vertices as low as possible, since then you can have more polygons in rendering.
• From artistic perspective, it is better to have artistic control on topology, since the topology of 2d representation of 3d objects can vary a lot.
• When you import a lot of small details as separate sprites, it is often nice to keep as many of them quads, since it is less vertex-intensive in a rendering queries.

Combining those factors, drawing the topology by hand and only for those sprites which need this may be the preferred scenario for the end-user needs. I already heard from original COA Tools reviews, that it has good instrument for rapid drawing of the mesh, once you get used to it. Maybe focus on polishing the hand-driven tooling could be a better approach.

---

However, if you still wish to make automesh, or at least auto outline, we can look at classic convex outline algorithms, which are very well documented. We don't have to use python libraries for that. However, it might be processor-intensive task for python. Is it possible to integrate a cpp-lib file into the plugin's package? We can take the only feature we need from a massive lib, and we really need to make it once, I believe.

I suppose, making concave meshes is a really hard task, and it would be easier to control just by creating a convex mesh and then erasing and reorganizing extra vertices by hand.

[Aodaruma]

Thank you for your opinion. Indeed, the feature of drawing mesh topology by hand is one of the appeals of coa_tools. I recall that there was a slight bug in coa_tools' hand-drawn mesh tool in the version of Blender that upgraded to Python 3.10, so I'd like to prioritize addressing that issue in development.

As for automesh, a prototype has already been created in a branch (though it still has some bugs), realized by installing opencv and executing its functions (such as cv2.findContours). However, I believe that installing external libraries in Blender carries risks, such as breaking dependencies with other add-ons. As you mentioned, it would be preferable to either implement the algorithm on the add-on side or integrate a cpp-lib file.