search

google / model-viewer

Easily display interactive 3D models on the web and in AR!
https://modelviewer.dev
Apache License 2.0
6.91k stars 816 forks source link

Object is black on some Android devices #679

Closed AytoMaximo closed 5 years ago

AytoMaximo commented 5 years ago

Description

The relevant THREE github issue on this case: https://github.com/mrdoob/three.js/issues/14137. It was opened more than year ago and there is still no solution. Model-view player based on the same THREE.js library, but it does not work on a lot of popular devices. For example, many Xiaomi phones (even flagman ones) can't work with it. Tested on Redmi 4A (Adreno 308), Mi A1 (Adreno 506), Mi6 (Adreno 540). More devices with black textures issue can be found by the link above.

Errors in the console:

Screenshot of our own test , but can also be produced on the main page with astronaut. ShooterScreenshot-22-18-07-19

Browser Affected

I guess all available for Android platform browsers are affected.

OS

Additional information

  1. The problem is in MeshStandartMaterial (https://threejs.org/docs/#api/en/materials/MeshStandardMaterial). We use difuse, normal, metallic and roughness maps with this shader and get black texture problem in our own project with THREE.js too. I think the astronaut in model-viewer inherits the same issue.
  2. From the documenation about MeshStandartMaterial we can get that it's about "physically based rendering (PBR) has recently become the standard in many 3D applications, such as Unity, Unreal and 3D Studio Max.". Ok, I'm Unity developer, so let's do the test.
    • Open Unity 2019.1
    • Create project with LightWeightPipeline
    • Choose shader for material object similiar to THREE's MeshStandartMaterial. I'll attach the screenshots from Unity and THREE editor. Shaders parametres are very similiar.
    • Make a build
    • That's it, rendering is fine!

Unity-PBR-LWP THREE-Editor-PBR

So, I supppose the phones are OK with PBR and can handle diffuse/normal/metallic/rougness shader. THREE's developers think that there is nothing they can do.

I hope google engineers can handle this issue together with THREE developers, because Web AR really depends on it.

cdata commented 5 years ago

@AytoMaximo thanks for reporting. I only glanced at the upstream Three.js issue, but it sounds like they decided that this is a WebGL / vendor-specific issue?

Are you seeing this problem on all devices, or on a specific Android device?

AytoMaximo commented 5 years ago

@cdata I've tested on Xiaomi devices, all of them have this issue.

Also, you can get a big list of devices from the https://github.com/mrdoob/three.js/issues/14137 thread:

They can't decide that it is a WebGL issue 'cas from the first post we see the following: "This doesn't appear to be a problem with all WebGL texturing, though -- https://fltr.world/ (simple texture demo implemented with three.js shaders) works fine, as does https://webglsamples.org/aquarium/aquarium.html "(c).

The first link is unreachable, however the aquarium is really fine.

Also, that's why it is not vendor-specific issue, because aquarium is fine, and my example with Unity is fine.

I also posted the same issue on Chromium bug tracker, they asked me produce a minimal test case that shows that there is a problem with handling of a valid shader on this phone. And suggested these links: https://github.com/KhronosGroup/WebGL , in particular the tests under https://github.com/KhronosGroup/WebGL/tree/master/sdk/tests/conformance/glsl/misc , can be used as a template.

But I'm not shader developer and don't understand what is going on there :(

It seems like guys really need some help, because they struggle this issue for a year or longer with no results. Moreover, I think the list of devices is very big and it's not over yet, so developing Google AR view with THREE.js library inherits a serious bug.

paluh commented 5 years ago

I was able to "fix" rendering on my "Moto G4" by passing precision: 'lowp' here: https://github.com/GoogleWebComponents/model-viewer/blob/6eb788f5417ca34155c282199cb92f63cae3507a/src/three-components/Renderer.ts#L75

This is suggested as a workaround in this thread:

https://github.com/mrdoob/three.js/issues/14137

Of course hardcoding this value doesn't seem like a good solution to the problem.

paluh commented 5 years ago

I'm not sure if my previous post adds any value to the discussion as @AytoMaximo is aware of this workaround as an active participant of the pointed (in my comment) threejs issue thread. @AytoMaximo should I delete the above workaround suggestion?

elalish commented 5 years ago

Would you be so kind as to upgrade to our just released 0.6 version and repro? We did a bunch of work in this release to address device-specific rendering issues and I'm hoping this is solved, but I don't have access to enough test devices here to be certain.

AytoMaximo commented 5 years ago

@elalish, sorry for the late answer, had plenty of work to do :) Unfortunately, now it's black everywhere. On my Windows PC (Chrome) and iOS device too. The same test link: https://image.faradise.ru/arcore.html Did we mess up something after the update? We turned on automatic updates for this mini-project. image

cdata commented 5 years ago

@AytoMaximo We have a similar report in #736

@elalish I'm tentatively creating a v0.6.1 milestone in case we discover a rendering regression

AytoMaximo commented 5 years ago

@cdata I've noticed a very funny thing: my Xiaomi Redmi 4A is not showing model in the scene anymore (before it was just black). The situation is the same in our test project and in model-viewer.glitch.me

image

elalish commented 5 years ago

@AytoMaximo Thanks for the info! I've looked at your model and I think I know why it's black: It has an entirely black AO (ambient occlusion) map: image I also noticed something strange with the shadow when I turned that on: image This leads me to believe something else is not quite right with this model - maybe triangle orientation? How was this model created?

One final question, @donmccurdy this model renders fine in your viewer, which makes me think one or the other of us has a bug. Are we somehow being more strict with our glTF parsing?

donmccurdy commented 5 years ago

Is there a link to the model?

My viewer applies some direct light, which isn't affected by AO. If model-viewer is only using indirect lights, that (combined with a black AO map) may be why it's dark.

elalish commented 5 years ago

Ah yes, that explains it! I used the test model link three posts up; you can pull the model out of that.

elalish commented 5 years ago

@AytoMaximo Until I manage to get a local repro of your vanished model case, would you be so kind as to expand that shader error and copy the results here for me? That should give me a clear idea of what's going wrong.

AytoMaximo commented 5 years ago

@AytoMaximo Thanks for the info! I've looked at your model and I think I know why it's black: It has an entirely black AO (ambient occlusion) map:

As I know, our standard map set (for WebGL, iOS and Android) is: diffuse, normal, metallic and roughness. I first faced black texture problem while developing our own THREE.js player. So, I'm sure I was using the map set mentioned above. Also, the models for Android include united metallic/roughness map and the black AO map in the Red channel.

Here is the config for our test case: <model-viewer ar src="https://237529.selcdn.ru//awespace/product/6190/8786.glb" alt="3d model Faradise" auto-rotate camera-controls background-color="#ffffff" ios-src="https://237529.selcdn.ru/faradise/product/6190/8786.usdz" ></model-viewer>

As you can see, there is .USDZ file included for iOS, where the model is also black. Also, iOS .USDZ has metallic+roughnesss maps, not AO. That's for sure.

We have our own big database of 3d models converted for all platforms. Everything is fine everywhere except Model-Viewer. Some Android devices render object as black (because of THREE.js issue) and all devices started to render object as black after the last model-viewer update. Maybe there is some issue with texture maps interpretation.

elalish commented 5 years ago

Also, the models for Android include united metallic/roughness map and the black AO map in the Red channel.

I think this is one of the problems. Why a black AO map? That means all environmental lighting is turned off (which is all we use now, for consistency of rendering).

AytoMaximo commented 5 years ago

@AytoMaximo Until I manage to get a local repro of your vanished model case, would you be so kind as to expand that shader error and copy the results here for me? That should give me a clear idea of what's going wrong.

Yes, sure :) image

model-viewer.js:18902 THREE.WebGLProgram: shader error: 0 35715 false gl.getProgramInfoLog WebGLProgram @ model-viewer.js:18902 WebGLPrograms.acquireProgram @ model-viewer.js:19306 initMaterial @ model-viewer.js:25562 setProgram @ model-viewer.js:25723 WebGLRenderer.renderBufferDirect @ model-viewer.js:24753 renderObject @ model-viewer.js:25466 renderObjects @ model-viewer.js:25436 WebGLRenderer.render @ model-viewer.js:25238 packMipmaps @ model-viewer.js:55894 generatePMREM @ model-viewer.js:55777 pmremPass @ model-viewer.js:56328 [loadGeneratedEnvironmentMap] @ model-viewer.js:56198 generateEnvironmentMapAndSkybox @ model-viewer.js:56171 (anonymous) @ model-viewer.js:57989 [updateEnvironment] @ model-viewer.js:57988 updated @ model-viewer.js:57967 updated @ model-viewer.js:58573 updated @ model-viewer.js:58455 performUpdate @ model-viewer.js:742 _enqueueUpdate @ model-viewer.js:676 async function (async) _enqueueUpdate @ model-viewer.js:665 _requestUpdate @ model-viewer.js:629 initialize @ model-viewer.js:484 UpdatingElement @ model-viewer.js:314 ModelViewerElementBase @ model-viewer.js:56575 FocusVisibleCoordinator @ model-viewer.js:58642 AnimationModelViewerElement @ model-viewer.js:56797 LoadingModelViewerElement @ model-viewer.js:58229 ARModelViewerElement @ model-viewer.js:57049 ControlsModelViewerElement @ model-viewer.js:57677 EnvironmentModelViewerElement @ model-viewer.js:57940 StagingModelViewerElement @ model-viewer.js:58556 MagicLeapModelViewerElement @ model-viewer.js:58449 (anonymous) @ model-viewer.js:58674

AytoMaximo commented 5 years ago

I think this is one of the problems. Why a black AO map? That means all environmental lighting is turned off (which is all we use now, for consistency of rendering).

I'm not android developer, so I can't properly answer this question, but it was the only way to make it work on android devices just like other platforms. We couldn't get the proper result in Android studio without this fix. Also, as I know, our android developer was also discussing this issue with Google.

However, what's wrong with USDZ on iOS? There are no any crazy fixes. But it's black in model-viewer.

elalish commented 5 years ago

We have no code to render a USDZ; all we do is pass it to QuickLook on iOS (hence the short transition into AR). If you're not in AR, then on iOS you're still looking at a glTF. If the AR one is black, then QuickLook doesn't like the USDZ either.

I did just notice that you have a warning about not supporting texture_float_linear, which I thought we did not require. I'm going to look into that.

AytoMaximo commented 5 years ago

If you're not in AR, then on iOS you're still looking at a glTF

Oh, I see. In AR the model is fine indeed. So, seems like we need to upgrade our glb model for the model-viewer.

I did just notice that you have a warning about not supporting texture_float_linear, which I thought we did not require.

This is one of the three errors (there is a list in my first post) inherited from the THREE.js library

elalish commented 5 years ago

Okay, I've verified that all three of those WebGL extension warnings are okay; three.js looks for each of them, regardless of whether they will be used. We intentionally use none of these so that rendering works without their presence. If you decide to include an AO map without data, ensure it is white, since it is multiplicative (1 has no effect).

Regarding your shader error, normally it outputs the the line on which the shader compiler had a problem. I'm not sure why it's not doing that here. I may need to get a local repro to debug. Does that only happen on the Xiaomi Redmi 4A?

AytoMaximo commented 5 years ago

Does that only happen on the Xiaomi Redmi 4A?

It's the only android device I have access to :( I'll ask around, but it's gonna take some time

AytoMaximo commented 5 years ago

Xiaomi Redmi Note 7 Pro (Adreno 512) - OK Honor V20 (Mali-G76 MP10) - OK Mi A1 (Adreno 506) - OK Mi6 (Adreno 540) - OK Redmi 3S (Adreno 505) - OK Redmi 6 (PowerVR GE8320) - OK

Xiaomi Redmi 4A (Adreno 308) - FAIL Sony Xperia Z (Adreno 320) - FAIL

It looks like old devices have some problems. Maybe we need some other reports about Adreno 3xx or 4xx. Just to be sure when the problem starts.

elalish commented 5 years ago

Great, so it seems the 0.6.0 update did fix at least the 506 and 540 models. If you can change the AO map to white and check that fixes the rendering, that would be helpful. If the only issue left is on old phones and can be fixed by switching to mediump (as stated above, is that still true?), then I would consider making that change here as long as we don't see major moire patterns from it.

AytoMaximo commented 5 years ago

If you can change the AO map to white and check that fixes the rendering, that would be helpful.

So, we changed AO to white and the chair become colorful. However, it's too bright :) image

I feel strange about the THREE.js material you are using.

I've imported .glb model to THREE.js editor. It's also black, but! because of no directional light was added to the scene. If I add a light, everything will be fine. Changing AO color or disabling it do nothing.

image image image image

Also, if I add black AO map to your astronaut, it'll be OK too: image

So, the question is: what kind of THREE.js material are you using in model-viewer? For example, MeshBasicMaterial does not need light to be visible (and it's OK if you use only diffuse map). But if you have a set of diffuse, normal, metallic and roughness, you need a MeshStandardMaterial, and this guy need lights in the scene.

image

So, that's why we didn't expect the AO map affects anything.

elalish commented 5 years ago

Thanks for the detailed analysis! Three.js does not apply AO to direct lighting, since this is a more basic lighting type. It is (and always has been) applied to environmental lighting, but you may not have noticed the difference since the render was dominated by the point lights. We recently removed the directional light from \, so all light is now environmental, which gives us a more consistent render because the two lighting types use different approximations. In fact the directional light was hiding a number of our own errors in environmental lighting which have since been corrected. In this case it was also hiding an AO error in the model.

elalish commented 5 years ago

As for brightness, please take a look at our exposure attribute, which provides the easiest way to adjust this.

AytoMaximo commented 5 years ago

So, I guess we need to find proper shade of gray color for AO map to make the model look not too bright :)

UPD: just noticed your comment about exposure - we’ll take a look, thank you!

Seems like old devices the only painful thing that remains...

donmccurdy commented 5 years ago

Yeah I wouldn't recommend using AO as a method of controlling overall brightness – if it's going to be a uniform color, you may as well omit AO entirely or use solid white. Anything else is taking you off a physically-plausible rendering path — nothing ever really gets zero indirect light — and lighting adjustments from there on out will likely be hacks.

Environmental lighting is higher quality than punctual/analytical lighting like directional lights. Changing the IBL or changing exposure should be enough to get realistic results. One could debate whether AO should affect IBL, but that's a thorny question and not the source of the problem here. :)

AytoMaximo commented 5 years ago

I also noticed something strange with the shadow when I turned that on:

I guess the wrong scale causes broken shadows. I've found out the chair was exported with some errors, so it's x100 large. Maybe it's need to be handled in model-viewer in some way, maybe not - just sharing the info :)

image

elalish commented 5 years ago

Thanks, that's very helpful! I was just cleaning up the shadow code, so I'll be sure to check that it works properly even at 100x scale in 0.7.

AytoMaximo commented 5 years ago

Thanks, that's very helpful! I was just cleaning up the shadow code, so I'll be sure to check that it works properly even at 100x scale in 0.7.

Sounds great! We have some problems with the scale while converting models to GLB. So, if model-viewer can handle such cases, it will be very helpful))

elalish commented 5 years ago

I plan to, but keep in mind, you definitely need proper scale for AR.

AytoMaximo commented 5 years ago

I plan to, but keep in mind, you definitely need proper scale for AR.

Yes, sure)

Deankovitch commented 3 years ago

I'm getting this "black faces error" also on pure Three JS project. In my project, the model is partially black. I'm using FBX exported from Maya. After checking everything from faces normals, to texture colors and UV smooth values, I decided to make a test recreating the material directly in Three JS. To my surprise, everything starts to render properly. So, for those are facing this black faces issue in ThreeJS project, here's a workaround

FBXloader.load( 'model.fbx', function ( object ) { 
object.traverse( function ( child ) {
            if ( child.isMesh ) {
                var newMat = new THREE.MeshStandardMaterial({
                    map: child.material.map,
                    metalnessMap: child.material.metalnessMap,
                    roughnessMap: child.material.roughnessMap,
                    normalMap: child.material.normalMap,
                    aoMap: child.material.aoMap,
                });
                child.material = newMat;
            }
        });
});

Unfortunately for the model viewer there isn't a workaround since it is some kind of a black box around the base Three JS

elalish commented 3 years ago

@Deankovitch What did you use to convert the FBX to glTF? That's what you'll want for model-viewer, and if it's generating black faces, please file a bug on that glTF exporter. Also make sure to check it with the Khronos glTF validator.