Large black blobs generated (possibly due to parallel faces)

[zardini123]

Hi ForestKatsch, thank you for such a great tool!

I'm trying to use your tool for terrain-like sculpts. These terrain-like models are not height map based, therefore I can give it overhangs and complicated geometry like holes.

Unfortunately, there are large black spots and blobs that are being formed. I noticed on your BlenderMarket FAQ that the black blob issue is known, and one should disable modifiers before baking. I gave that a shot. Didn't fix. I adjusted all possible parameters before baking. I set the distance parameter to very high, and very low values. I ensured good topology, including merging duplicate vertices, ensuring correct normals. Nothing changed. My theory is it has to deal with the fact some faces are parallel to, but facing away from, one another.

I also want to clarify that the geometry is essentially like a "sheet" (as like most terrain models), therefore not enclosed. The shape was made by taking a plane, making it huge, subdividing it, using dynotopo to sculpt, and passing it through a re-mesher.

Heres some screenshots for clarification: Other side of this cliff: An overhang. Large bands of black on vertical faces, but not as much on upside down faces. Other side of overhang.

[ForestKatsch]

Are the vertices all merged? I add a small "fuzz factor" to the ambient occlusion rays; otherwise, the ray cast from the vertex would immediately hit its own face and end.

[zardini123]

Are the vertices all merged?

Yes, I ensured all vertices are merged.

I add a small "fuzz factor" to the ambient occlusion rays; otherwise, the ray cast from the vertex would immediately hit its own face and end.

Is this currently how it operates, or is this for future implementations?

I have no idea how AO calculations work, but I was wondering, do the rays go in the direction of the normal? If they did, I wouldn't expect them to make these blobs...

[ForestKatsch]

At the moment, any ray being cast at the same object starts a little outside (in the direction of the normal), and every other ray is cast starting a little inside the object. This is so that two objects that are directly against one another occlude each other, but objects don't occlude themselves.

I'm seeing some hard edges on the AO in your model, and I think that only occurs if Auto Smooth is on, or (maybe) if the mesh isn't set to smooth. Can you try setting smooth (w -> Set Smooth) and disabling Auto Smooth (in the mesh settings panel), then see if that solves the issue?

[zardini123]

I went under Object Data Properties > Normals and found Auto Smooth was already set to off. I tried enabling it, the angle field was greyed out, so I went to Geometry Data and removed Custom Split Normals Data.

Even after ensuring Auto Smooth was disabled, custom split normals were removed, and re-setting normals to smooth (it originally wasn't in the screenshots), and the blobs still persisted.

My mesh is majority quads, but some areas do have one or two triangles, and/or some n-gons. When enabling viewing of Normals in edit mode, the per-vertex normal of multiple of these triangles had lines coming from inside the middle of the triangle, not facing the normal (heh) direction! I was so confused! But then when I enabled vertex editing, I realized every triangle (not quad) has loose vertices in the middle of it!! In Edit Mode, I removed these loose vertices using Mesh > Clean Up > Delete Loose.

After doing that clean up, I re-ran Vertex Oven on it, and the issue persisted! At least I understand that my re-mesher creates those unwanted vertices...

[zardini123]

I'd also like to point out that after going through all these steps (including smooth shading), the vertex colors due to the AO now do not have any hard edges anymore. The AO bleeds into surrounding faces like intended. Though of course, the issue with the blobs is still present.

[ForestKatsch]

Hmmm, that should have worked. Do you have a small reproduction .blend file so I can troubleshoot? Thanks!

[zardini123]

Hmmm, that should have worked. Do you have a small reproduction .blend file so I can troubleshoot? Thanks!

Yeah sure, I attached it to this comment. I sectioned off the main trouble area that I have a screenshot of earlier. I ensured that after isolating the geometry that the AO issue still persisted, and it did.

If you open vertex paint you can see the bake I see on my end. That'd be interesting if it bakes differently on your end...

AOBlackBlobTest.blend.zip

[ForestKatsch]

I can reproduce this on my end. I'll keep this issue updated.

[ForestKatsch]

Fixed in 32e4ffc.

[ForestKatsch]

This has been fixed in v0.1.9; here's the release: https://github.com/ForestKatsch/VertexOven/releases/tag/v0.1.9

[zardini123]

This has been fixed in v0.1.9; here's the release: https://github.com/ForestKatsch/VertexOven/releases/tag/v0.1.9

I gave the new version a shot, and it fixed the areas I have screenshots previously. Though, a blob I haven't documented exists. In another angle: If you want me to send another file, I can.

[ForestKatsch]

So the fundamental bug here is that the offset value I chose was too small for big faces, and due to floating-point error would be on the wrong side of the mesh. I bumped it up a little bit (to half a millimeter), but your mesh is so large that even that has enough floating-point error to manifest itself.

I can add a configurable "margin" value, which would be set to a higher number when baking meshes with large faces; or a smaller number when baking small meshes. I'll add this feature soon.

[zardini123]

Would it be possible to make the margin value proportional to the size of each face, so the margin will always work regardless of face size?