How to dynamically change normal, metallic and roughness with Textures?

[yakivmospan]

Hey guys,

In our case, we have pure model.obj mesh, without texture references in it. And baseColorMap.png, normalMap.png, metallicMap.png, roughnessMap.png files that we want to use as Texture's for our mesh. The set of textures can vary for one mesh, so, for example, we can have one mesh and 8 png files (two flavors of one model).

To load my obj file dynamically, I can first convert it to sfb, no probs with that. But what can I do, to apply my textures to it dynamically? That is completely possible to do with SceneKit& ArKit on iOS (including mesh generation from obj file) and what we've already done there.

We saw #41 , #197 and #112 issues. According to those, with texture, we can change only the base color of our mesh, using MaterialFactory.makeOpaqueWithTexture(context, baseColorMapTexture). Thats because MaterialFactory uses default .sfm material definition?

Can we somehow create Material for a custom compiled *.mat definition, so we will be able to apply another textures to Material with setTexture("normalMapTexure") method and then set it to Renderable? At least that is possible to do with filament, in this sample you are loading material definition from file. Can we create Material with filament and pass it to Sceneform ?

Would be great to have something like this in API :

MaterialFactory.createFromMaterialDefinition("location/to/material.mat")

We also tried to convert our obj to fbx, where default fbx *.sfm material definition has already implemented normalMap, metallicMap and roughnessMap parameters. Unfortunatelly as per #41 , we were not able to change any of them using renderable.getMaterial().setTexture("normalMap", texture);. Maybe thats because there are no mentions about textures in .fbx model?

Thanks in advice, Yakiv.

[romainguy]

Currently the only way to load a custom material is to compile it into a .sfb with a mesh (say a single triangle :) and to call getMaterial() after loading it. This should let you change the textures dynamically.

[yakivmospan]

Thank you @romainguy , we found that @michaelvogt is doing this in #390 and started to implement our material definition for .obj textures already. Will update this post if we succeed on it.

Are you planning to expose custom material creation feature, sometime in the future? Also, is there any chance to have obj runtime download capabilities (the same as we have with glb/gltf currently)?

[malik-at-work]

We are tracking the feature request to open up material creation at https://github.com/google-ar/sceneform-android-sdk/issues/393

[yakivmospan]

@malik-at-work thanks, subscribed. Also, as @romainguy mentiond using the fake object, we've manage to get our custom material:

material {
    name : "Textured material",
    parameters : [
        {
           type : sampler2d,
           name : baseColorMap
        },
        {
            type : sampler2d,
            name : normalMap
        },
        {
            type : sampler2d,
            name : metallicMap
        },
        {
            type : sampler2d,
            name : roughnessMap
        }
    ],
    requires : [
        uv0
    ],
    shadingModel : lit,
    blending : opaque
}

fragment {
    void material(inout MaterialInputs material) {
        vec3 normal = texture(materialParams_normalMap, getUV0()).xyz;
        material.normal = normal * 2.0 - 1.0;

        prepareMaterial(material);

        material.baseColor = texture(materialParams_baseColorMap, getUV0());
        material.metallic = texture(materialParams_metallicMap, getUV0()).r;
        material.roughness = texture(materialParams_roughnessMap, getUV0()).r;
    }
}

One more question, can we change the default value for created parameters?

[yakivmospan]

We've prepared a sample project that demonstrates how to change textures dynamically. Hope it will help others to answer questions that we got.

[HEMAL-PATEL]

We've prepared a sample project that demonstrates how to change textures dynamically. Hope it will help others to answer questions that we got.

Thank For great demo. But I want to change the color of the added object at runtime by selection of colors from a list of colors in alert dialog or elsewhere. So, is it possible to change only the color of the object at runtime?

If possible then can we change the color of some perticular part of the object?

Please help me. Thank you in advance.

[yakivmospan]

Hey @HEMAL-VES , yes it is possible to change only color texture, you can use only baseColorTexture from sample project. Or you can play with MaterialFactory.

To change the color of a particular object part, you got few options:

• Use sub-meshes in your 3D models. This will require you to change 3D model, and this is the only one option if you will choose to go with MaterialFactory. Also using this way, you will face another issue with getting sub-meshes names.

• Use different texture files, for different colors. There you can choose what part of the model shall have a different color.

[romainguy]

You can also create a custom material that lets you change a color multiplier (this feature should be supported by gltf).

[harsha-main]

Hello, I don't understand methods like what getUV0 does, do you needsome pre requisite knowledge to understand these? I just need to set culling to null for my Material object as mentioned in my question #448. Can someone provide help here? @malik-at-work can you provide any help here?

[HEMAL-PATEL]

Hey @HEMAL-VES , yes it is possible to change only color texture, you can use only baseColorTexture from sample project. Or you can play with MaterialFactory.

To change the color of a particular object part, you got few options:

• Use sub-meshes in your 3D models. This will require you to change 3D model, and this is the only one option if you will choose to go with MaterialFactory. Also using this way, you will face another issue with getting sub-meshes names.
• Use different texture files, for different colors. There you can choose what part of the model shall have a different color.

Hi @yakivmospan

I checked your example which you have posted to change texture dynamically. But in that example, you have used texture image which you are using to change the texture of the object. That's a great one. But I did not want to this in my project. I just want to change the color of an object by selecting a color from color picker dialog. Which change color dynamically of the object.

Is there any way to change the color of the object dynamically at runtime without using the texture image file Or Custom Material Reference?

[yakivmospan]

@HEMAL-VES Hey, as I wrote in previous post, take a look at MaterialFactory class, there is makeTransparentWithColor method, that creates Material from passed color. Then you can apply created Material to your Renderable.

Or as @romainguy mentioned, you can play with color multiplier. Cheers

[vortice3D]

Hi Yakiv:

In order to use metallic and roughness for FBX, with a focus on economy of resources (textures), I wonder if it's possible to have a unique map for both (as we have in glTF).

The idea here would be to use the three channel in a more standard PBR way:

• channel red for ambient occlusion (not applicable for Sceneform);
• channel green for roughness;
• channel blue for metallic.

In the sfa file, for certain material we'll have:

   ...
  {
    metallic: 1.0,
  },
  {
    metallicMap: 'Torre_occlusionRoughnessMetallic',
  },
  {
    roughness: 0.1,
  },
  {
    roughnessMap: 'Torre_occlusionRoughnessMetallic',
  },
  ...

The material fragment (extract) would be like this:

...
material.roughness=texture(materialParams_roughnessMap,getUV0()).g;
material.metallic=texture(materialParams_metallicMap,getUV0()).b;
...

Does this will work?

[Dosssik]

Currently the only way to load a custom material is to compile it into a .sfb with a mesh (say a single triangle :) and to call getMaterial() after loading it. This should let you change the textures dynamically.

Hi guys

Is it possible to change textures for 3d-model with animation (.sfb compiled from .fbx) in runtime? In our case, we have to download 3d-models from the server, separately download textures, and finally apply 'em to models. As far as i can see renderable.getMaterial.setTexture() still doesn't work, unfortunately. I saw the solution from @yakivmospan (thank's a lot for Medium article!) but looks like it is suitable for .obj model, not for .fbx with animations.

Is there any working solution?

Tnx, Andrey

[ChanduMnS]

Hi @yakivmospan , Thanks for creating your post. I've followed your post on medium. My project deals with the similar issue. I've been provided with models [.fbx] without any textures applied to it by default. I've given different swatch images showcasing the fabric of the material. I'm able to render the model as it is. There are 100's of combination of colours for the same product. I'm not familiar with Kotlin because of which I'm not able to follow the code. I'm not able to understand the way you created the custom material. If you could explain or suggest any other tutorial, I'll appreciate that. Thanks in Advance. Chandu.

[ChanduMnS]

Currently the only way to load a custom material is to compile it into a .sfb with a mesh (say a single triangle :) and to call getMaterial() after loading it. This should let you change the textures dynamically.

Hi guys

Is it possible to change textures for 3d-model with animation (.sfb compiled from .fbx) in runtime? In our case, we have to download 3d-models from the server, separately download textures, and finally apply 'em to models. As far as i can see renderable.getMaterial.setTexture() still doesn't work, unfortunately. I saw the solution from @yakivmospan (thank's a lot for Medium article!) but looks like it is suitable for .obj model, not for .fbx with animations.

Is there any working solution?

Tnx, Andrey

I've a similar use case. Your help will be appreciated @yakivmospan . Thanks