GLB model textures are all completely black

[nuudles]

• [X] Review the documentation: https://virocore.viromedia.com/
• [X] Search for existing issues: https://github.com/viromedia/virocore/issues
• [X] Use the latest ViroCore release: https://virocore.viromedia.com/docs/releases
• [X] Turn off Instant Run in Android Studio

Environment

Please provide the following information about your environment:

1. OS: Mac
2. Version: 1.14.0
3. Device(s): Samsung Galaxy S9, Pixel 2

Description

We are using several GLB models in an iOS app and they have been working great for us. We are now trying to use ViroCore with the same GLB models and we are having issues with them appearing completely black (see screenshot below). We have tested with converted FBX files and they look fine, so lighting isn't the issue.

The models also appear great in the GLTF viewer and validator: https://gltf-viewer.donmccurdy.com/

Reproducible Demo

One of the GLB files that we tested with can be found here: https://www.dropbox.com/s/hq6vgogqzmbk90k/boeing_body.glb?dl=0

[nuudles]

@dthian @dam00n We would really appreciate some feedback regarding this ticket as it is obviously a major blocker for us if we cannot get these models to render correctly.

[dam00n]

Can you provide a code snippet of how you are displaying the model in the scene?

[nuudles]

@dam00n The model is loaded and put in what we call a stage node:

class StageNode(private val viroContext: ViroContext) : Node() {
    private val modelNode = Object3D()

    init {
        setUpNode()
    }

    private fun setUpNode() {
        addChildNode(modelNode)
        modelNode.loadModel(viroContext,
            Uri.parse("file:///android_asset/models/boeing_body.glb"),
            Object3D.Type.GLB, object : AsyncObject3DListener {
                override fun onObject3DLoaded(object3D: Object3D, type: Object3D.Type) {
                    val box = modelNode.boundingBox
                    val delta = Vector(box.maxX - box.minX, box.maxY - box.minY, box.maxZ - box.minZ)
                    val scale = Math.min(Math.min(0.5f / delta.x, 0.5f / delta.y), 0.5f / delta.z)
                    modelNode.setPosition(
                        Vector(
                            -(box.minX + delta.x / 2.0f),
                            -(box.minY + delta.y / 2.0f),
                            -(box.minZ + delta.z / 2.0f)
                        )
                    )
                    modelNode.scalePivot =
                        Vector(
                            box.minX + delta.x / 2.0f,
                            box.minY + delta.y / 2.0f,
                            box.minZ + delta.z / 2.0f
                        )
                    modelNode.setScale(Vector(scale, scale, scale))
                }

                override fun onObject3DFailed(s: String) {
                    Log.e("onObject3DFailed", s)
                }
            })
    }
}

The stage node is put in a scene that is set up like this:

        val scene = ARScene()
        scene.setAnchorDetectionTypes(EnumSet.of(ViroViewARCore.AnchorDetectionType.PLANES_HORIZONTAL))
        viroView.scene = scene

        val rootNode = scene.rootNode

        val light = AmbientLight(Color.parseColor("#606060").toLong(), 3000f)
        light.influenceBitMask = 3
        rootNode.addLight(light)

        val omniLight = OmniLight()
        omniLight.color = Color.WHITE.toLong()
        omniLight.position = Vector(0f, 0f, 0f)
        stageNode.addLight(omniLight)

        val spotLight = Spotlight()
        spotLight.influenceBitMask = 1
        spotLight.position = Vector(0.0, 5.0, 0.0)
        spotLight.castsShadow = true
        spotLight.attenuationEndDistance = 7f
        spotLight.attenuationStartDistance = 4f
        spotLight.direction = Vector(0.0, -1.0, 0.0)
        spotLight.intensity = 6000f
        spotLight.shadowOpacity = 0.35f
        stageNode.addLight(spotLight)

        anchorNode.addChildNode(stageNode)
        stageNode.setPosition(Vector(0f, 0f, -1.0f))
        rootNode.addChildNode(anchorNode)

[dthian]

Thanks @nuudles, by any chance do you have the .gltf file as well for the model?

[nuudles]

@dthian I included the GLB link in the ticket when I created it: https://www.dropbox.com/s/hq6vgogqzmbk90k/boeing_body.glb?dl=0

[dthian]

Sorry @nuudles, i mean the corresponding .gltf file before it was compressed to .glb, do you have that? (I'm also looking for converters online to see if i can convert back to gltf)

[nuudles]

@dthian I see. I don't think we have the original gltf but I have used GLB to GLTF converters in the past

[dthian]

Thanks @nuudles, I was able to grab a .gtlf parsed version of it. There seems to be two issues: 1) Could you verify with the original model, if both the metallic and roughness texture supplied to the model is as to spec? (They seem to just be completely 0) 2) The same goes fo the occlusion texture as well - could you check if it's to spec? (It also seems to be completely 0)

[nuudles]

@dthian Here is a GLTF version of the model: https://www.dropbox.com/s/shpy3shoi9nl1kl/boeing_body_2.zip?dl=0

[nuudles]

@dthian Have you had a chance to look at the GLTF? In that one it looks like the textures all specify indices for a buffer specified by the external PNG files. This seems to be supported by the spec.

[dthian]

Hey @nuudles, as mentioned above, i was actually asking if the data within the textures above were within spec as desired. In particular:

1) The occlusion texture is using the "fuselaje__Phong_occlusionRoughnessMetallic.png" as its source, where the png is completely black. And as the gltf spec states:

The occlusion values are sampled from the R channel. Higher values indicate areas that should receive full indirect lighting and lower values indicate no indirect lighting.

Thus, since the above occlusion texture is completely black (rgb is 0,0,0), then your pbr model will receive no indirect lighting - is this desired? (Might explain why the model is black).

2) Finally, it looks like the phong normals (fuselaje__Phong_normal.png) of your model are all facing the same direction - this desired? This can have a huge impact on lighting. This will explain why the model looks so dark, except when the phone is tilted in one direction (because your normals are all facing the same direction).

Note: If i disable both occlusion texture and normal textures, the model renders as expected with pbr textures and an environment light map as shown below.

[dthian]

Update: I am mistaken For # 2 - this is actually an empty normal map. We are currently investigating why providing an empty normal map would result in a black model.

Also, from the code sample above, is it intentional not to use environment lighting and instead control direct lighting manually? (The visual outlook would be different, compared to rendering indirect light with pbr).

[nuudles]

Thank you. Please let us know if you find anything with regard to the empty normal map. It's worth mentioning that so far ViroCore is the only renderer that has had a problem rendering these models.

Regarding environmental lighting, we wanted to minimize the factors to ensure that there were no other issues causing the model to display incorrectly so we are using a simple set of lights for testing purposes.

[dthian]

Yeah, the above boeing_body.glb is a strange model, and for your use case above it sounds like you don't want to use environment lighting at all (else you won't have that occlusion texture that effectively nulls out all indirect lighting). So it's a strange use case that we are currently investigating.

Using normal textures should have worked as well. For example, testing with gLTF's sample FlightHelmet model renders fine, and it has several normal maps that work.

As such, we are currently looking into why using a basic normal map results in a black model, for a pbr model that is desired to be rendered without environment lighting - to confirm is this right? (We would like to try get as close as to your desired prod conditions as possible).

To conclude: 1) Know that it is possible to mitigate the above by simply removing the "occlusionTexture" lines within the .gltf file. After doing that, you should see the model render with direct lighting.

2) We are currently looking into why the basic normal map results in a black model.

3) Even after the basic normal map issue is fixed, lighting the model with directly lighting may not result in the desired visual output. (You can test that by using the mitigations above) - is this desired? For the most realistic and best visual output, i strongly recommend using environment lighting to render pbr models, as that produces the most natural look.

[nuudles]

@dthian Yes, we understand the benefits of the environment lighting. Once the model is appearing correctly, we will be playing with the lighting in general, but the number one priority is to have the model appearing. We have dozens of these models that we are already using in our iOS app, so our hope is that we can keep them as-is to render properly using ViroCore.

[nuudles]

As of right now we don't have any plans to use environment lighting, but that may change as the requirements evolve.

[radvani]

Hi @nuudles, FYI we've corrected this issue (see attached image -- the model renders correctly without any modifications) and it will be available in our next release. If you need a hot-fix earlier than that, please send us an email at support@viromedia.com.

[dam00n]

This issue has been fixed in the latest release, ViroCore v1.15

Release Notes -> https://virocore.viromedia.com/docs/releases