Submeshcount always 0, assign new Texture with filament

[nPhil01]

I first tried this with Sceneform 1.16, but since it can not handle filament version greater than 8.1, I tried this version.

I need to render GLB models, run their animations and change the texture.

Changing textures with the Sceneform functionality does not work, because the submeshcount is still always zero. So I tried changing the textures with the filamentAsset, which does not work either. I tried getting help with this on github and on stackoverflow, with limited success.

Any help?

[nPhil01]

It seems the filamentAsset provided by the RenderableInstance of a Node is not complete. I included filament's modelViewer and loaded the GLB there, too.

engine = Engine.create();

modelViewer = new ModelViewer(engine);
Buffer buffer = readAsset("avocado.glb");
modelViewer.loadModelGlb(buffer);

So i can get the filamentAsset from the renderableInstance

AnchorNode anchorNode = new AnchorNode(anchor);

// attaching the anchorNode with the ArFragment
anchorNode.setParent(arCam.getArSceneView().getScene());

// attaching the anchorNode with the TransformableNode
TransformableNode model = new TransformableNode(arCam.getTransformationSystem());
model.setParent(anchorNode);

// attaching the 3d model with the TransformableNode
// that is already attached with the node
model.setRenderable(modelRenderable);
model.select();

// Saving the object as filament asset
// -> Possibility to run animations and change textures
filamentAsset = model.getRenderableInstance().getFilamentAsset();

and the filamentAsset provided by the filament modelViewer

filamentAsset2 = modelViewer.getAsset();

After getting the entitites

RenderableManager renderableManager = engine.getRenderableManager();

int[] entities = filamentAsset.getEntities();
Log.d("Entities", String.valueOf(entities.length));

int[] entities2 = filamentAsset2.getEntities();
Log.d("Entities2", String.valueOf(entities2.length));

I loop through the entitites and try to get the materialInstances

for (int entity: entities) {
    Log.d("EntityName", filamentAsset.getName(entity));
    int instance = renderableManager.getInstance(entity);
    Log.d("Instance", String.valueOf(instance));
    if (instance == 0)
        continue;
    MaterialInstance materialInstance = renderableManager.getMaterialInstanceAt(instance, 0);
    Log.d("MaterialInstance", materialInstance.getName());
    materialInstance.setParameter("baseColorMap", textureBC, textureSampler);
}

for (int entity: entities2) {
    Log.d("EntityName2", filamentAsset2.getName(entity));
    int instance = renderableManager.getInstance(entity);
    Log.d("Instance2", String.valueOf(instance));
    if (instance == 0)
        continue;
    MaterialInstance materialInstance = renderableManager.getMaterialInstanceAt(instance, 0);
    Log.d("MaterialInstance", materialInstance.getName());
    materialInstance.setParameter("baseColorMap", textureBC, textureSampler);
}

The logs should be the same, but:

D/Entities: 1
D/Entities2: 1
D/EntityName: Avocado
D/Instance: 0 
D/EntityName2: Avocado
D/Instance2: 1
D/MaterialInstance2: 2256_Avocado_d

As you see, the Instance is 0 at the first one, the asset that was created with renderableInstance.getFilamentAsset().

I am not sure if this is the reason for my problems, but is there any workaround for this? Unfortunately, the modelViewer from filament is not usable (I have not found a way yet) with the arFragment of Sceneform, only witht the Android surfaceView.

[ThomasGorisse]

As you see, the Instance is 0 at the first one, the asset that was created with renderableInstance.getFilamentAsset().

This is a normal result. Why using :

if (instance == 0)
    continue;

You just skip the targeted MaterialInstance doing that.

Anyway, even if it's not a suitable solution for coming versions of Sceneform wich will try to let you manage filament assets directly, here is (I hope) your solution :

ModelRenderable.builder()
  .setSource(this, Uri.parse("avocado.glb"))
  .setIsFilamentGltf(true)
  .build()
  .thenAccept(
      modelRenderable -> {
          modelRenderable.getMaterial().setTexture("baseColorMap",
            Texture.builder()
              .setSource(yourBitmap)
              .setSampler(Texture.Sampler.builder()
                      .setWrapMode(Texture.Sampler.WrapMode.REPEAT)
                      .build())
              .build());
      });

Sceneform 1.8.5 will apply all the filament calls for you. But, just to let you know, I'm working on making ModelRenderable beeing directly a FilamentAsset.

[nPhil01]

This is a normal result. Why using :

if (instance == 0) continue; You just skip the targeted MaterialInstance doing that.

I am doing this because for instance == 0 I get this error messge: A/libc: Fatal signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0x0 in tid 12678 (e.myapplication), pid 12678 (e.myapplication)

The application crashes in this situation.

When I try using your code

            ModelRenderable.builder()
                    .setSource(this, R.raw.avocado)
                    .setIsFilamentGltf(true)
                    .build()
                    .thenAccept(
                            modelRenderable -> {
                                try {
                                    Log.d("setModel", "loadMaterial");
                                    modelRenderable.getMaterial().setTexture("baseColorMap",
                                            Texture.builder()
                                                    .setSource(mainBitmap)
                                                    .setSampler(Texture.Sampler.builder()
                                                            .setWrapMode(Texture.Sampler.WrapMode.REPEAT)
                                                            .build())
                                                    .build()
                                                    .get());
                                    Log.d("setModel", "loadedMaterial");
                                } catch (ExecutionException | InterruptedException e) {
                                    e.printStackTrace();
                                }
                                Log.d("setModel", "addModel");
                            });

nothing happens. Only the first Log is logged. When I check at this point for submeshcount, it is still zero, so the getMaterial() method should not work here, too, right? Also, I need to use the get() method because build() is just returing the completeableFuture.

[ThomasGorisse]

There is a big missing part concerning the the FilamentAsset instanciation since the 1.16 moved to gltfio. I'm working hard on a freshly complety new version using all the benefits of latests filament versions. Since the release is out, can you try to apply all your texture modification after the first Renderable instance rendering = after the the node placement on the scene?

[nPhil01]

You mean like this:

@Override
protected void onCreate(Bundle savedInstanceState) {
    super.onCreate(savedInstanceState);
    setContentView(R.layout.activity_main);

    // If the system supports AR, set up the scene
    if (checkSystemSupport(this)) {
        getSupportFragmentManager().beginTransaction()
                .add(R.id.arCameraArea, ArFragment.class, null)
                .commit();
        loadTexture(R.raw.avocado_basecolor);
    }
}

@Override
public void onAttachFragment(@NonNull Fragment fragment) {
    super.onAttachFragment(fragment);

    if (fragment.getId() == R.id.arCameraArea) {
        arCam = (ArFragment) fragment;

        // Listening for taps on recognised planes
        arCam.setOnTapArPlaneListener((hitResult, plane, motionEvent) -> {

            Log.d("hitResult", hitResult.getHitPose().toString());

            // Render the object only if no other objects was rendered yet
            Anchor anchor = hitResult.createAnchor();

            ModelRenderable.builder()
                    .setSource(this, R.raw.avocado)
                    .setIsFilamentGltf(true)
                    .build()
                    .thenAccept(modelRenderable -> addModel(anchor, modelRenderable))
                    .exceptionally(throwable -> {
                        AlertDialog.Builder builder = new AlertDialog.Builder(this);
                        builder.setMessage("Somthing is not right" + throwable.getMessage()).show();
                        return null;
                    });
        });
    }
}

private void loadTexture(int resourceId) {
    Resources resources = getResources();

    Bitmap bitmap = BitmapFactory.decodeResource(resources, resourceId, null);
    mainBitmap = bitmap;

    Log.d("bitmapHeight", String.valueOf(bitmap.getHeight()));
    Log.d("bitmapWidth", String.valueOf(bitmap.getWidth()));
}

private void addModel(Anchor anchor, ModelRenderable modelRenderable) {
    // Creating a AnchorNode with a specific anchor
    AnchorNode anchorNode = new AnchorNode(anchor);

    // attaching the anchorNode with the ArFragment
    anchorNode.setParent(arCam.getArSceneView().getScene());

    // attaching the anchorNode with the TransformableNode
    TransformableNode model = new TransformableNode(arCam.getTransformationSystem());
    model.setParent(anchorNode);

    // attaching the 3d model with the TransformableNode
    // that is already attached with the node
    model.setRenderable(modelRenderable);
    model.select();

    try {
        Log.d("setModel", "loadMaterial");
        modelRenderable.getMaterial().setTexture("baseColorMap",
                Texture.builder()
                        .setSource(mainBitmap)
                        .setSampler(Texture.Sampler.builder()
                                .setWrapMode(Texture.Sampler.WrapMode.REPEAT)
                                .build())
                        .build()
                        .get());
        Log.d("setModel", "loadedMaterial");
    } catch (ExecutionException | InterruptedException e) {
        e.printStackTrace();
    }
}

Here, the texture should be set right after the rendering. Still the same error, submeshcount is 0. The newest release is the 1.18.5 one, right? (in gradle build file: implementation 'com.gorisse.thomas.sceneform:sceneform:1.18.5')

[ThomasGorisse]

Fixed here: 5c2f6ce24e2cc88490f065e53cf78eada8f0cc7a