search

gkjohnson / three-gpu-pathtracer

Path tracing renderer and utilities for three.js built on top of three-mesh-bvh.
https://gkjohnson.github.io/three-gpu-pathtracer/example/bundle/index.html
MIT License
1.34k stars 133 forks source link

Support multimaterial meshes #546

Open dellamonica opened 7 months ago

dellamonica commented 7 months ago

When trying to use the pathtracer to render a scene that includes many meshes (mostly boxes) with internal and face materials, the tracer seems to use the wrong materials (images below). Curiously, if I do a roundtrip GLTF conversion (using GLTFExporter and GLTFLoader) and render the loaded GLTF, the results are correct!

Expected behavior

The pathtracer should properly render the original scene, the extra steps of GLTF conversion are probably just changing the materials and meshes in a way that causes the bug to not manifest.

Screenshots and Repro Model

Original:

original

Pathtraced from original: some doors are rendered with their interior material rather than surface material. It looks like some pieces are rendered with the metallic material that belongs to the pull handle while some pull handles are rendered in flat colors rather than metal. A "blue" drawer shows up in the corner as well (this is the color of some edges). Overall the materials have been shuffled.

pathtrace_from_original

After doing the export GLTF => load and import GLTF model, the results are correct:

pathtrace_from_gltf

Since the GLTF export actually works, I'm not sure how I could create a repro model.

Platform:

gkjohnson commented 7 months ago

Since the GLTF export actually works, I'm not sure how I could create a repro model.

This can't be looked at without some kind of reproduction case. If you can't share a model please make a small project showing the issue.

dellamonica commented 7 months ago

Would serializing the scene with .toJSON work for you? For the full scene, the JSON file is ~185Mb. I loaded the saved scene* and could reproduce the bug with it.

If necessary, I could create a small repro project that loads the scene from the JSON and runs the Pathtracer, but that would take some time and perhaps is not necessary if you already have your own repro environment.

Thank you very much for your help and for the wonderful projects you have here.

gkjohnson commented 7 months ago

I'd prefer a smaller reproduction with a minimal scene and code that shows the issue. Dealing with 185MB of scene and material data is going to be a lot to dig through.

dellamonica commented 7 months ago

OK, I'll try a much smaller scene, but I'll need some time to prepare code + data. Please bear with me.

dellamonica commented 7 months ago

Here's a small project. repro.zip

There are fewer textures to go around, but none of the faces got the right texture. image image

gkjohnson commented 7 months ago

Thanks! The project is using multiple materials per mesh which is not currently supported.

Also please note in the future that it's much better to provide a minimal example without the need for other unrelated client dependencies like react or typescript. It makes things much more difficult to follow otherwise.

dellamonica commented 7 months ago

OK, so the GTLF exporter is splitting the mesh and that is why it worked then.

Do you plan to add support for this? If not, do you have any idea how to easily split the meshes on my end? I looked into https://github.com/mrdoob/three.js/blob/master/examples/jsm/exporters/GLTFExporter.js but it is ~3k lines and the processMesh function calls other functions and local state, so it doesn't look very simple to extract just the splitting logic from there.

Thanks again.

gkjohnson commented 7 months ago

Do you plan to add support for this?

It will probably be added at some point though I have other priorities at the moment. If you'd like to make the contribution yourself I'm happy to review and merge a PR, though I'm not immediately sure of what the extent of the changes that would be needed are. Changes to three-mesh-bvh would likely need to be made, as well.

If this is for a professional project and you'd like to fund the development and priority of this feature for the project I'm happy to discuss that, as well.

If not, do you have any idea how to easily split the meshes on my end?

The three.js forums would be the best place to ask a question like this.

dellamonica commented 7 months ago

Thanks for the info. It is not a critical feature, I can work with the GLTF step for now. The project is indeed professional and I've just sent a humble sponsorship (from my company's account). There's no need to prioritize this feature though, what you've done has already been extremely useful to me.