Normal mapping with custom material

[vs-dos]

Hello.

How can I use a normal maps with custom material? I noticed that for some reason they are flipped, and when I try to change the way they rendered nothing happens.

Here's fragment part of material definition I used.

fragment {
    void material(inout MaterialInputs material) {
        vec3 nm =  normalize(texture(materialParams_normal, getUV0()).xyz) * 2.0 - 1.0;
        material.normal = nm * vec3(1.0,-1.0, 1.0);
        prepareMaterial(material);
        material.baseColor = materialParams.baseColorFactor * texture(materialParams_baseColor, getUV0());
        float3 metallicRoughnessSampler = texture(materialParams_metallicRoughness, getUV0()).xyz;
        material.metallic = metallicRoughnessSampler.b;
        material.roughness =  metallicRoughnessSampler.g;
    }

Result I got.

[romainguy]

Flipped in what way? Are the texture coordinates flipped or are the normals themselves pointing the wrong way? If they are pointing the wrong way, you should flip the Z coordinate instead of the Y coordinate (+Z points outside of the surface).

[vs-dos]

All textures coordinates are correct, this model looks as expected in Unity, and Scenekit. I tried to set normals as mentioned here like

normal = texture(materialParams_normalMap, getUV0()).xyz;
material.normal = normal * 2.0 - 1.0;

but it works the same way as shown at screensot above. I noticed that if I try to put any value different from 0 in normal.Y, some faces are always black. What is stored here? As far as I understood the idea 2DSampler provides texture. In each RGB texel of this texture there are encoded XYZ vector. The next step is: material.normal = normal * 2.0 - 1.0; By doing this step we convert color components(0 - 1) to vector (-1 - 1). In theory, it should work, but when I run this, some faces are black. Should I manually convert normals from tangent space to world space with TBN? Or is there any information how your shader calculates normals and tangent space?

[romainguy]

You don't need to convert from tangent space to world space manually, this is done for you by the shaders.

The TBN matrix is constructed this way:

shading_tangentToWorld = mat3( normalize(vertex_worldTangent), normalize(vertex_worldBitangent), normalize(n));

And applied this way:

shading_normal = normalize(shading_tangentToWorld * material.normal);

In the normal vector, +z points outside the surface, x and y are the tangent/bitangent coordinates.

BTW can you make sure that your texture is not loaded as an sRGB texture in Sceneform? The normal map must be treated as linear data. We've seen bugs related to this before.``

[vs-dos]

Thank you so much for provided information. I've found a workaround for this issue. I put values only in material.normal.xz. So I use this material.normal.xz = normalize(texture(materialParams_normal, getUV0()).xz * 2.0 - 1.0); It looks like the shader have to infer the third component from the other two. To compare result I converted this model using default gltf shader, and here is the result The left ones is default gltf material, right ones is custom material. I see the normals are applied. however, I do not understand why bitangents should always be zero.

[romainguy]

Would you mind showing me your normal map? You shouldn't have to do this.

[vs-dos]

Sure.

[romainguy]

Most of the green values in your normal maps are ~127, which translates to 0 after the * 2 - 1 operation. I've tried loading your normal map in a simple test app that uses the renderer directly and I get the correct result (see attached picture). This test app does the following:

material.normal = texture(materialParams_normalMap, getUV0()).xyz * 2.0 - 1.0;

It seems like your image may not be loaded properly. Do you get the same issue when rendering the asset on device or is it only in the Android Studio preview window? Could you also please double check and make sure the normal map is not marked as a color texture/loaded in sRGB?

Thanks!

[vs-dos]

As I understand, the way texture is loaded depends on "usage" option of 2dsampler. I checked this option. Here's .mat I tested with.

material {
    name: "Custom material",
    parameters: [
        {
            //BaseColor rgba factor
            name: "baseColorFactor",
            type: float4
        },
        {
            //BaseColor map sampler
            name: "baseColor",
            type: sampler2d
        },
        {
            //Normal map sampler
            name: "normal",
            type: sampler2d,
            usage: "Normal"
        },
        {
            //MetallicRoughness map sampler
            name: "metallicRoughness",
            type: sampler2d,
            usage: "Data"
        }
    ],
    requires: [
        position,
        uv0,
        color,
        tangents
    ],
    shadingModel: lit,
    blending: opaque,
    transparency : twoPassesOneSide,
}

fragment {
    void material(inout MaterialInputs material) {
        vec3 normal = texture(materialParams_normal, getUV0()).xyz;
        material.normal = normal * 2.0 - 1.0;
        prepareMaterial(material);
        material.baseColor = materialParams.baseColorFactor * texture(materialParams_baseColor, getUV0());
        vec3 mr = texture(materialParams_metallicRoughness,getUV0()).rgb;
        material.metallic = mr.b;
        material.roughness = mr.g;
    }
}

Here's what i got.

[gstanlo]

What if you tried using a test texture to verify the shader is working? Maybe something like this that I found searching for a normal map test:

Note that the differences between the heads should be in the green channel, and you should be consistent with whether green means up or down in the shader and textures you use.

[vs-dos]

I checked this with several normal maps, it's okay. The main problem is that there are some logic that sets the local normal vector of the objects back face to (0,0,0) and it becomes black. So I've tested this with some normal maps and the result was equal. Most likely there is something wrong in shader logic when it converts vectors like (0,0,-1) to world space. Or something overrides vertex normal values. I checked that thing with default cube exported from Maya. With normal map its back face was always black. Here are resources I used. cube.zip

[gstanlo]

I looked at the gltf in the zip you provided. I'm still investigating, but this might be a bug with the Sceneform importer or converter. The issue is regarding generated tangents. It should be okay to not have tangents on your model, as the converter should generate them for you, but they seem to be getting corrupted somehow.

As a temporary workaround, can you try exporting the model with tangents?

[vs-dos]

I researched this thing. Looks like it's render bug, I created my own model format with geometry inside, and get renderable using RenderableDefinition, and result is the same, tangents of models are calculated wrong way. So with RenderableDefinition there is no such option to set tangents manually.

[vs-dos]

@gstanlo Is this possible to add this feature? I did not used sfb converter in example above, probably tangents are not calculated with RenderableDefiition. Being able to add tangents with Vertex.builder could help solve this problem.

[gstanlo]

This looks like a bug -- I think being able to specify tangents on the RenderableDefinition sounds reasonable.

Are you able to proceed with a model that has tangents specified on it? I would need to be a FBX or glTF/2.0 format. I believe an OBJ file with texcoords should work too.

[vs-dos]

Hello @gstanlo can we expect this issue be solved in upcoming release?

[jordanyim1]

Hey @gstanlo can you explain how using a model with tangents can fix this problem?
I've tried this and it doesn't work:

• Export FBX in blender with tangents
• Convert to SFB with custom material
• Add normal map into SFA When I apply a normal map it, it is still broken.

This looks like a bug -- I think being able to specify tangents on the RenderableDefinition sounds reasonable.

Are you able to proceed with a model that has tangents specified on it? I would need to be a FBX or glTF/2.0 format. I believe an OBJ file with texcoords should work too.

[vs-dos]

Hello! Custom materials were released about six month ago. However, it's still not possible to use normal mapping. Is there any information on whether this will be fixed?