Texture preservation for mesh simplification(LOD)

[DVLP]

Texture preservation for mesh simplifier

Here's a model reduced to 40% of original vertex count with textures preserved. With this you can generate LOD instead of baking it.

This modification of SimplifyModifier adds texturing support and is replacing face UVs on removed vertex to UV coordinates of remaining vertex. More or less works. I can make a PR unless you want to improve this first.

https://codesandbox.io/s/5018onxy1p

Update:

Details: Added a check if percentage of texture coordinates change is disproportionately higher than vertex shift distance(percentage of distance to diameter of bounding sphere) then no UV change will be applied on face at removed vertex position.

https://codesandbox.io/s/jvy6x7r3my

This seems to improve the end result and strange texture stretching but I don't think it's fixing the root cause of the problem.

Three.js version

• [ x] Dev
• [ x] r92

Browser

• [x] All of them
• [ ] Chrome
• [ ] Firefox
• [ ] Internet Explorer

OS

• [x] All of them
• [ ] Windows
• [ ] macOS
• [ ] Linux
• [ ] Android
• [ ] iOS

[Mugen87]

More or less works.

I can see a lot of artifacts around her head.

[DVLP]

I managed to update with a fix while you were writing about your findings :) I put a workaround in place. Not sure if that's the right approach but the improvement is big.

Update: Note that now I've increased optimisation level to 60% reduction so comparing with above image will not be accurate. Better compare both links as I also changed the first one to 60% reduction for fair comparison, or change the new link optimisation level to 0.5 to compare with the image above

[Mugen87]

Can you please explain your approach? Or link to a document/paper that describes your technique?

Her hair looks still problematic...

[DVLP]

Check again(same link): https://codesandbox.io/s/jvy6x7r3my

No papers to support this and I'm experimenting. Maybe reinventing the wheel. This is a bit of shooting in the dark to prevent optimiser assigning coordinates from other "slice" of texture to moved vertex.

To prevent unexpected changes I'm comparing distance of vertex change to texture coordinates change and if the difference is bigger than 5% it won't change UVs but will leave them and texture will stretch on currently processed triangle. Stretch will bring less unexpected results than moving to another texture area and bringing artifacts. This check can be configurable but with this particular model I find 5% to work the best so for now leaving hardcoded.

Basically: allow large UV coordinates change when large vertex position change occurs and prevent large UV coordinates change when vertex shift was much smaller.

My knowledge ends here. For now. Result so far(60% vertices removed):

[Mugen87]

Just for reference: https://github.com/songrun/SeamAwareDecimater

This is actually a complicated topic 😊

[DVLP]

It is, but earlier today I made a much more straighforward approach than my last one. https://codesandbox.io/s/23p6j1ow9j

When computing cost of edge collapse I'm also checking if next to both vertices are any material coordinates different than others(basically checking if all UVs around a vertex are the same - in my opinion this is the easiest way to define is there's a texture seam on vertex). If any UV around vertex is different than others it's being added to edge collapse cost.

Texture preservation is much better now. Now it's a matter how strongly texture preservation should influence cost. This creates preserve shapes vs preserve texture borders tradeoff.

Still, if you compare with the previous link the other method seems to be better in other areas

[sjlynch]

Is there any chance someone could create a stand alone build of this (i.e. does not require webpack and/or babel)? I am having a lot of trouble transpiling this to vanilla JS. It would be great to try this out on some of my project's models.

I'm working on an automatic LOD feature (LODs would be cached server side) that I want to apply to different model formats (obj, gltf, and fbx mostly) with varying complexity (anywhere from 1000 verts to 500k) so I may be able to provide some valuable feedback and bug reports once I get this working.

Edit: Got it working, had to edit the javascript files manually to remove all the webpack and babel stuff, wasn't fun. Maybe I should learn how to use these tools correctly :).

This modifier works very well! I have only tested one 700k vertex multiple material fbx model so far. It was reduced to 155k with no noticeable texture issues. I am impressed!! It's so fast too, only 30 seconds for this large model! It might even be faster than Blender's decimate tools.

I'll test more models over the weekend and hopefully give some useful feedback. Thanks for working on this @DVLP Let me know if there are specific types of models you would like me to try.

[DVLP]

Hi sjlynch thanks for taking time to test this method! This code is actually out of date and I rewrote the whole thing to work directly on BufferGeometry completely bypassing any legacy Geometry implementation. Also I added multi threading in webworkers so on large meshes it's exponentially faster(2 threads are 4 times faster, but 4 threads are just 8 times faster than 1) Here's the topic with the latest version https://github.com/mrdoob/three.js/issues/5806 and here's the latest version https://codesandbox.io/s/xrlplpmr1q

[sjlynch]

@DVLP That is the version I'm using. So far I have only tested the model mentioned above. I have another FBX file of similar complexity that I'll try next.

BTW, far future, I may attempt to write a tensorflow app that can use your tool's output as training data and learn to decimate any mesh. Theoretically, this should be faster and could lead to an optional automatic LOD feature for threejs. Socializing this idea might light a fire under my butt :)

[DVLP]

@sjlynch You should be able to drop your FBX file directly on the doll and it will be loaded. There's handling of dropped files implemented in the sandbox.

Good luck making an optimiser with Tensor flow but this sounds more like a lifestyle than a side project :)

[yazancash]

Thanks for your code. i made very small updates and applies on my project. it works perfectly. here is example.

[donmccurdy]

Would we be interested in adding an example showing mesh simplification using meshoptimizer? It will preserve all vertex attributes. That's what I'm using in glTF Transform as well.

[Mugen87]

Can mesh optimizer be used at runtime or is it required to prepare the asset via node.js CLI first?

[donmccurdy]

It can be used at runtime, and is a 15kb (without gzip) JavaScript+WASM file:

https://github.com/zeux/meshoptimizer/blob/master/js/meshopt_simplifier.module.js

The main 'gotcha' is that vertices need to be appropriately welded before running simplification. BufferGeometryUtils.mergeVertices may be sufficient, I'll have to check on that.

I would encourage people to do simplification offline if possible, in tools like Blender, gltfpack, or glTF Transform. Neither welding nor simplifying are cheap on large meshes. But runtime simplification will work too.

[Moebits]

@DVLP It seems like Geometry has been removed, would it be possible to adapt the original code to work on BufferGeometry instead? The one that you did with web workers is too complicated to adapt for my needs.

[DavianDiazM]

Hi @DVLP, I need to ask you a huge favor and ask you certain things about your code, I am developing a computer engineering thesis on optimization of 3D scenes and your code would save me a lot, I have problems understanding and using it, sorry if I ask a lot for your lack of availability of time, but you would help me a lot with this and I need it, I am a beginner and there are things that are difficult for me to understand about the code, if you are available to ask me a question, write to me by telegram, this is my user @Brendan_R, I adjust to your time availability, but I really need to use this code, thanks

[AddMeFastBot]

This may helps you, it's written in ThreeJS to simplyfy models: https://rigmodels.com/3d_LOD.php?view=5OOOEWDBCBR630L4R438TGJCD

[DavianDiazM]

@AddMeFastBot that code uses an obsolete version of three.js, which uses the geometry class, thanks for your attention

[mrdoob]

I think we can close this one now?

[Mugen87]

After merging https://github.com/mrdoob/three.js/pull/26875, the geometry retains its texture coordinates in the simplest possible way. AFAICT, this issue is about adding a more sophisticated approach to SimplifyModifier which computes texture coordinates such that texture distortions is mitigated as good as possible.