isl-org / spear

SPEAR: A Simulator for Photorealistic Embodied AI Research
MIT License
224 stars 15 forks source link

`kujiale_0000` scene on Linux has some strange material behavior/reflections #243

Open RachithP opened 1 year ago

RachithP commented 1 year ago

Here are some screenshots of kujiale_0000 scene. This was generated on a Ubuntu 22.04 machine, with the v0.3.0 executable and v0.3.0 pak file.

Screenshot from 2023-08-09 07-51-43 Screenshot from 2023-08-09 07-52-02

mikeroberts3000 commented 1 year ago

This looks similar to the MSPresets/Megascans issue we were having in the lead-up to the 0.3.0 release (https://github.com/isl-org/spear/issues/209, https://github.com/isl-org/spear/issues/210). Does the scene look like this in the editor? Is the Quixel Bridge plugin enabled when you're building the executable and the paks? Does this happen when you build kujiale_0000 directly into the executable without going through build_paks.py? Have you double-checked for old pak files that are being mounted? It is possible that everything is correct, but you're mounting an old pak file also, and the old pak file isn't referring to some assets in the correct way.

rwang15 commented 1 year ago

I think the problematic mateials here are kujiale materials. The megascan material (floor, couch, pillow) looks fine.

RachithP commented 1 year ago

Few observations:

This looks similar to the MSPresets/Megascans issue we were having in the lead-up to the 0.3.0 release (https://github.com/isl-org/spear/issues/209, https://github.com/isl-org/spear/issues/210). Does the scene look like this in the editor?

I get the same behavior in Editor mode. However, the materials affected here seem to be different from the ones observed in #209, #210, as mentioned by @rwang15. Screenshot from 2023-08-10 12-48-38

Is the Quixel Bridge plugin enabled when you're building the executable and the paks?

Yes, it's enabled.

Does this happen when you build kujiale_0000 directly into the executable without going through build_paks.py?

Yes, this occurs even in this situation. I wouldn't be surprised as we can see this behavior in editor mode, I'd think this would propagate to paks irrespective of the method used to build it.

Have you double-checked for old pak files that are being mounted? It is possible that everything is correct, but you're mounting an old pak file also, and the old pak file isn't referring to some assets in the correct way.

Yup, I'm not mounting any pak file here, instead I'm directly placing the pak file in Content/Paks folder in the executable's dir to be sure.

mikeroberts3000 commented 1 year ago

I was able to reproduce the problem by: (1) inserting a default cube into debug_0000; (2) enabling Nanite on the cube in the static mesh editor (note that I made a copy of the cube so I wasn't directly modifying an asset that belongs to the engine); (3) assigning MI_BasicShapeMaterial_Inst_Orange to the cube; and (4) setting MI_BasicShapeMaterial_Inst_Orange to be two-sided.

It appears that Nanite does not support two-sided materials on Linux in UE 5.2. I'm not sure how this ever worked in our previous experiments. It is possible that two-sided materials used to be supported on Linux in 5.1, but were subsequently broken in 5.2. According to the 5.1 release notes, Nanite is supposed to support two-sided materials (https://docs.unrealengine.com/5.1/en-US/unreal-engine-5.1-release-notes). Regardless, we will need to decide whether we want Nanite enabled, or two-sided materials enabled, because it appears that we can't have both on a single object on Linux.

mikeroberts3000 commented 1 year ago

Update: I was able to reproduce the problem in UE 5.1 (I happened to still have UE 5.1 on my Linux workstation). So I think two-sided materials + Nanite + Linux have never worked. Maybe we enabled Nanite on some of the kujiale_0000 meshes recently, which would explain why we weren't seeing this visual artifact until recently.

RachithP commented 1 year ago

I was able to reproduce the problem by: (1) inserting a default cube into debug_0000; (2) enabling Nanite on the cube in the static mesh editor (note that I made a copy of the cube so I wasn't directly modifying an asset that belongs to the engine); (3) assigning MI_BasicShapeMaterial_Inst_Orange to the cube; and (4) setting MI_BasicShapeMaterial_Inst_Orange to be two-sided.

Can confirm that the problem is reproducible with these steps.

rwang15 commented 1 year ago

Given that it is an Engine bug and we might expect it to be fixed at some point, maybe we could still enable both Nanite and two-sided material as default, and temporaily disable one of them with code at runtime. This will keep our scenes in good shape without the need to revisit the scene and makes extensive change. Having some special if-else code is a bit unpleasant but less painful than changing the scene in my opinion.

rwang15 commented 1 year ago

As discussed in the meeting, we should edit the raw assets and enable two-sided material only when necessary (meshes that would not render correctly with single-sided material, such as foliage, etc).