[major improvement] Implementing Voxel Remesh

[thelazyone]

Currently the limit of remeshing for 3D sculpts is given by the fact that a 3D voxel-remeshed model tends to be huge.

As a real-world example from my works, a single building can easily reach 10 GB of faces after a 0.02mm spaced remeshing. Unfortunately such fine remeshing is necessary to maintain sharp edges that otherwise get blurred away.

The current remesh modifier in Blender uses OpenVDB (https://github.com/blender/blender/blob/main/source/blender/blenkernel/intern/mesh_remesh_voxel.cc#L185), being able to implement something similarly efficient would be a great boon for any complete remesh workflow.

As a bonus, a voxel remesher would be capable to handle multiple non-connected meshes (joining them together) which is a common situation in the case of 3D sculpting for printing purpose.

[thelazyone]

A full implementation in C++ is available here. It is indeed a massive amount of work, but it still would be precious. https://github.com/davidstutz/mesh-voxelization

[dima634]

FIY: I started to work on a VDB-like data structure for storing volumetric data. It is the first step to getting volumetric modelling functionality in the library. And eventually, I think there will be voxel-based remeshing. https://github.com/dima634/baby_shark/tree/vdb

[thelazyone]

FIY: I started to work on a VDB-like data structure for storing volumetric data. It is the first step to getting volumetric modelling functionality in the library. And eventually, I think there will be voxel-based remeshing. https://github.com/dima634/baby_shark/tree/vdb

This is amazing! I'll follow with interest, would like to try and integrate the whole baby_shark into my (very prototypal) online visualizer eventually ( https://github.com/thelazyone/3d-external-visualizer ) When you get to the voxel part i'd love to contibute!

[thelazyone]

@dima634 I've checked now the massive amount of new stuff in the branch - great job! It appears it's still not usable from the outside though - let me know if I can be of any help!

( https://github.com/dima634/baby_shark/pull/19 )

[dima634]

@dima634 I've checked now the massive amount of new stuff in the branch - great job! It appears it's still not usable from the outside though - let me know if I can be of any help!

( #19 )

At the current stage, it is possible to create a signed distance field from an implicit function and convert it to mesh using marching cubes. Now, I am working on mesh to SDF conversion. The current implementation can create some artefacts in mesh, but more or less is working. You can check it out in the voxel example. However, you are right it is not ready for outside world yet.

[thelazyone]

The voxel example works fine (aside from an issue being discussed in the #19 ) but i wanted to apply the decimation directly instead of writing it as stl and re-loading. I'm very confident that it will happen soon... and again, would love to lend a hand if possible!

[dima634]

@thelazyone I am finalizing voxel re-meshing and preparing to merge the vdb branch to the master. I would appreciate some feedback. You can find an example of how to use voxel re-meshing here.

[thelazyone]

@thelazyone I am finalizing voxel re-meshing and preparing to merge the vdb branch to the master. I would appreciate some feedback. You can find an example of how to use voxel re-meshing here.

Cheers there,

The remeshing works smoothly! The only feedbacks I could have are:

1. Each example refers to different stls that are not present in the repo. Committing a simple Susan or even a cube would make the examples run out of the box

2. I tried to connect the remeshing to a simple decimation, but I'm struggling to find a way to convert the PolygonSoup into a CornerTable.

   fn main() {
   type Mesh = PolygonSoup<f32>;
   
   let mut reader = StlReader::new();
   let mesh: Mesh = reader
       .read_stl_from_file(Path::new("head.stl"))
       .expect("Read mesh");
   
   let mut remesher = VoxelRemesher::default().with_voxel_size(0.02);
   let remeshed: PolygonSoup<f32> = remesher.remesh(&mesh).unwrap();
   
   // This obviously doesn't work. :3
   let remeshed_corner_map: CornerTableF = remeshed.try_into::<CornerTableF>().unwrap(); 
   
   let decimation_criteria = ConstantErrorDecimationCriteria::new(0.01f32);
   let mut decimator = EdgeDecimator::new().decimation_criteria(decimation_criteria);
   decimator.decimate(&mut remeshed_corner_map);
   
   StlWriter::new()
       .write_stl_to_file(&remeshed, Path::new("remeshed_small.stl"))
       .expect("Write mesh");
   }

3. The remeshing seems to create slight artifacts in the convex angles: comparing it with the clean mesh and blender it appears that the remeshing creates a rift between the parts, where Blender simply smooths it. Not sure which one is preferable, in truth.

Hopefully it's useful feedback. This would be a killer tool for people looking to remesh and simplify 3D files for 3D printing!

[thelazyone]

@dima634 I would also love to integrate this with a visualizer I started a while ago. It's a super barebones thing, mostly an idea, but I think it's something that is not out there already, no?

https://github.com/thelazyone/3d-external-visualizer

TL;DR: a web-based listener that visualizes any bunch of polys that are provided via websocket, and a basic protocol to send such polygons (or points?) with Rust, Python or whatever. Sometimes it's tedious to open and refresh .stl files, and one wants to do some 3D manipulation without having to set up a GUI.

I spent WAYS too little time on this, but would love your opinion on whether it's something useful, or if it exists already (100 times better, i'd imagine).

[dima634]

@thelazyone I am finalizing voxel re-meshing and preparing to merge the vdb branch to the master. I would appreciate some feedback. You can find an example of how to use voxel re-meshing here.

Cheers there,

The remeshing works smoothly! The only feedbacks I could have are:

1. Each example refers to different stls that are not present in the repo. Committing a simple Susan or even a cube would make the examples run out of the box
2. I tried to connect the remeshing to a simple decimation, but I'm struggling to find a way to convert the PolygonSoup into a CornerTable.

fn main() {
    type Mesh = PolygonSoup<f32>;

    let mut reader = StlReader::new();
    let mesh: Mesh = reader
        .read_stl_from_file(Path::new("head.stl"))
        .expect("Read mesh");

    let mut remesher = VoxelRemesher::default().with_voxel_size(0.02);
    let remeshed: PolygonSoup<f32> = remesher.remesh(&mesh).unwrap();

    // This obviously doesn't work. :3
    let remeshed_corner_map: CornerTableF = remeshed.try_into::<CornerTableF>().unwrap(); 

    let decimation_criteria = ConstantErrorDecimationCriteria::new(0.01f32);
    let mut decimator = EdgeDecimator::new().decimation_criteria(decimation_criteria);
    decimator.decimate(&mut remeshed_corner_map);

    StlWriter::new()
        .write_stl_to_file(&remeshed, Path::new("remeshed_small.stl"))
        .expect("Write mesh");
}

4. The remeshing seems to create slight artifacts in the convex angles: comparing it with the clean mesh and blender it appears that the remeshing creates a rift between the parts, where Blender simply smooths it. Not sure which one is preferable, in truth.

Hopefully it's useful feedback. This would be a killer tool for people looking to remesh and simplify 3D files for 3D printing!

1. Good point. I will add simple box geometry to run an example on it.
2. I will add a conversion between PolygonSoup and CornerTable in future versions of the library. For now, you can do something similar to this.

   let faces = remeshed.vertices().map(|v| *remeshed.vertex_position(&v)).collect::<Vec<_>>();
   let indexed_faces = merge_points(&faces);
   let mesh = CornerTable::from_vertices_and_indices(&indexed_faces.points, &indexed_faces.indices);

   3/4. Hard to say what is happening there. Looks like something minor and the mesh surface after remeshing is still pretty close to the original.

@dima634 I would also love to integrate this with a visualizer I started a while ago. It's a super barebones thing, mostly an idea, but I think it's something that is not out there already, no?

https://github.com/thelazyone/3d-external-visualizer

TL;DR: a web-based listener that visualizes any bunch of polys that are provided via websocket, and a basic protocol to send such polygons (or points?) with Rust, Python or whatever. Sometimes it's tedious to open and refresh .stl files, and one wants to do some 3D manipulation without having to set up a GUI.

I spent WAYS too little time on this, but would love your opinion on whether it's something useful, or if it exists already (100 times better, i'd imagine).

Personally, I mostly use desktop apps to inspect meshes (MeshLab, MeshMixer and even default Windows 3D viewer). The only web-based viewer I am aware of is THREE.js editor. But it requires some basic knowledge of THREE.js and is not really user-friendly. I guess it was created mostly for demo purposes and not as an alternative to the existing 3D viewers.

[thelazyone]

Regarding the remeshing (using 1 and 2) i'll try to do some further tests and check the resulting outputs against what Blender does. Blender is really good at it all, but between the remesh and the decimation there are a few Python steps that make the whole process painfully slow.

The three js editor seems cool! But yeah, the scope of the visualizer would be to add some websocket (or something else) API that you can call via code and simply update the thing without having to click anything. A bit like a canvas.show() but without having to setup a whole local GUI.

Again, not sure if it's something that someone would find useful or if it's just me. :)

[thelazyone]

@dima634

The snippet of code worked and the remesh happened. A few notes:

1. I had to change the max_err value of the simplification from 0.01 to 500 to make it work. I tried resizing the head (thus changing the remeshing resolution as well) but the result was a crash entered unreachable code: Input must be invalid or non-manifold.

2. The remeshed model had several large holes, probably due to 3.

   

3. Inspecting the remeshed surface, and comparing it to a Blender voxel remesh, the topology is indeed a bit more cluttered.

   

4. Blender infamously automatically merges together any overlapping vertices, so I checked the model in Meshmixer, and it seems OK, but with scattered holes here and there:

   

5. Passing the remeshed file to MS 3D Builder (which always fixes any mesh, for some reason) the decimation (called through simplification.rs) works fine.

EDIT: head.zip Added head.stl for reproducibility.

[dima634]

@thelazyone I believe most of the problems you described are fixed in the latest version of the vdb branch.

There were a couple of bugs causing the problems you described:

• a bug with decimation introduced after switching from Point3 to Vector3 in the code
• my marching cubes implementation was creating non-manifold edges and vertices that caused panic and holes

Now, simplification should work fine with lower max error values and MC should not produce non-manifold meshes or meshes with holes.

Inspecting the remeshed surface, and comparing it to a Blender voxel remesh, the topology is indeed a bit more cluttered.

It is just the nature of the marching cubes algorithm. I guess Blender is using some kind of dual contouring for meshing that produces meshes with a bit better triangle quality. But MC is considered to produce better quality meshes (no self-intersections, non-manifold edges/vertices). Mb in future I will add an alternative meshing algorithm but for now, I think MC is good enough.

[thelazyone]

Excellently done. The current output mesh of the voxel remesh is absolutely comparable with Blender.

I've tried the whole cycle of remeshing+decimating, and it appears that now the bottleneck of quality is due to the decimation algorithm. I detailed this in #22 .

Thanks for the amazing job here!

[thelazyone]

As a fun note - working with voxel remeshing in Blender - I just noticed it creates some intersecting/reversed faces artifacts!

So maybe it's not the ideal final goal afterall! :3

[dima634]

@thelazyone Recently, I was playing with the Dual Contouring meshing algorithm and the resulting mesh seems to be pretty close to the Blender output. If you are interested here is the branch with updated code. However, the downside is that DC produces self-intersections.

[thelazyone]

That's very interesting!

I ended up writing a blunt tool to remove the self intersections for my models. It's kind of important for 3D printing (it isn't, but some of the slicers notify errors and that's not nice), but now my models have weird uneven faces every now and then.

Seems you're gonna be our only hope!