Android sample gltf-viewer crashed when using multiple instances

[oceanii]

⚠️ Issues not using this template will be systematically closed.

Describe the bug Android crashes when rendering using multiple engine instances. I tested and verified some of the machines based on 1.38.0 version, mobile phones such as vivo S17 Pro, vivo S15 Pro crash. mobile phones such as vivo S17t, vivo S15, vivo X90 Pro no crash and normal operation.

To Reproduce Steps to reproduce the behavior:

1. create multiple ModelViewer
2. bind each ModelViewer to a SurfaceView
3. run in the Android device above

Expected behavior multiple instances can run normally.

Screenshots

Logs the complete log has been added to the attachment log.txt

10-26 12:45:01.594 21182 21227 I libc : debuggerd signal invoked signal:11 10-26 12:45:01.594 21182 21227 I libc : debuggerd requestDump:0 NO_NEW_PRIVS:0 --------- beginning of crash 10-26 12:45:01.594 21182 21227 F libc : Fatal signal 11 (SIGSEGV), code 2 (SEGV_ACCERR), fault addr 0x7c3ace1d30 in tid 21227 (mali-cpu-comman), pid 21182 (.filament.gltf2) 10-26 12:45:01.595 21182 21227 I libc : clone child process pid:21254 10-26 12:45:01.596 21182 21227 I libc : debuggerd pseudothread crash_process: 21227 10-26 12:45:01.988 21256 21256 F DEBUG : Softversion: PD2284A***.W10.V000L1 10-26 12:45:01.988 21256 21256 F DEBUG : Time: 2023-10-26 12:45:01 10-26 12:45:01.988 21256 21256 F DEBUG : * ** 10-26 12:45:01.988 21256 21256 F DEBUG : Build fingerprint: 'vivo/PD2284/PD2284:13/TP1A.220624.014/compiler08221110:user/release-keys' 10-26 12:45:01.988 21256 21256 F DEBUG : Revision: '0' 10-26 12:45:01.988 21256 21256 F DEBUG : ABI: 'arm64' 10-26 12:45:01.988 21256 21256 F DEBUG : Timestamp: 2023-10-26 12:45:01.670618401+0800 10-26 12:45:01.988 21256 21256 F DEBUG : Process uptime: 3s 10-26 12:45:01.988 21256 21256 F DEBUG : Cmdline: com.google.android.filament.gltf2 10-26 12:45:01.988 21256 21256 F DEBUG : pid: 21182, tid: 21227, name: mali-cpu-comman >>> com.google.android.filament.gltf2 <<< 10-26 12:45:01.988 21256 21256 F DEBUG : uid: 10292 10-26 12:45:01.988 21256 21256 F DEBUG : tagged_addr_ctrl: 0000000000000001 (PR_TAGGED_ADDR_ENABLE) 10-26 12:45:01.988 21256 21256 F DEBUG : signal 11 (SIGSEGV), code 2 (SEGV_ACCERR), fault addr 0x0000007c3ace1d30 10-26 12:45:01.988 21256 21256 F DEBUG : x0 b400007ceaaf62d0 x1 0000000000000000 x2 0000007c3ace4790 x3 0000007b63356b18 10-26 12:45:01.988 21256 21256 F DEBUG : x4 0000000000000048 x5 0000000000000000 x6 0000000000000000 x7 0000e33a0000dfce 10-26 12:45:01.988 21256 21256 F DEBUG : x8 b400007c3ace1d30 x9 0000000000000160 x10 0000000000000060 x11 0000000000000010 10-26 12:45:01.988 21256 21256 F DEBUG : x12 0000000000000001 x13 0000000000000080 x14 0000000000000000 x15 0000000000000001 10-26 12:45:01.988 21256 21256 F DEBUG : x16 0000007bbf2859b8 x17 0000007e8d567f80 x18 0000007b58616000 x19 b400007c9aa15500 10-26 12:45:01.988 21256 21256 F DEBUG : x20 b400007c9aa154f0 x21 b400007ceaaf62c0 x22 0000000000000001 x23 0000000000000001 10-26 12:45:01.988 21256 21256 F DEBUG : x24 b400007ceaaf6160 x25 0000007b63356cb0 x26 0000007b63356ff8 x27 0000007b6a2fe000 10-26 12:45:01.988 21256 21256 F DEBUG : x28 0000000000000002 x29 0000007b63356c50 10-26 12:45:01.988 21256 21256 F DEBUG : lr 0000007bbe5b14a0 sp 0000007b63356b80 pc 0000007c3ace1d30 pst 0000000080001000 10-26 12:45:01.988 21256 21256 F DEBUG : backtrace: 10-26 12:45:01.988 21256 21256 F DEBUG : #00 pc 00000000000a1d30 [anon:scudo:primary] 10-26 12:45:01.988 21256 21256 F DEBUG : #01 pc 000000000175e49c /vendor/lib64/egl/mt6895/libGLES_mali.so (BuildId: c7abf1eb1bf27a24) 10-26 12:45:01.988 21256 21256 F DEBUG : #02 pc 000000000175ce38 /vendor/lib64/egl/mt6895/libGLES_mali.so (BuildId: c7abf1eb1bf27a24) 10-26 12:45:01.988 21256 21256 F DEBUG : #03 pc 000000000175cd38 /vendor/lib64/egl/mt6895/libGLES_mali.so (BuildId: c7abf1eb1bf27a24) 10-26 12:45:01.988 21256 21256 F DEBUG : #04 pc 0000000000108b80 /apex/com.android.runtime/lib64/bionic/libc.so (__pthread_start(void)+224) (BuildId: cfc293be733954571ce0dc79a9917039) 10-26 12:45:01.988 21256 21256 F DEBUG : #05 pc 000000000009bf60 /apex/com.android.runtime/lib64/bionic/libc.so (__start_thread+64) (BuildId: cfc293be733954571ce0dc79a9917039)

Smartphone (please complete the following information):

• Device: vivo S17 Pro, vivo S15Pro
• OS: Android 13, Android 12
• Platform: MT6896, MT6895T

Additional context sample code： layout: <?xml version="1.0" encoding="utf-8"?> <LinearLayout xmlns:android="http://schemas.android.com/apk/res/android" android:id="@+id/simple_layout" android:layout_width="match_parent" android:layout_height="match_parent" android:orientation="vertical">

<SurfaceView
    android:id="@+id/main_sv"
    android:layout_width="match_parent"
    android:layout_height="0dp"
    android:layout_weight="1" />

<SurfaceView
    android:id="@+id/main_sv2"
    android:layout_width="match_parent"
    android:layout_height="0dp"
    android:layout_weight="1" />

java: /*

• Copyright (C) 2020 The Android Open Source Project
• Licensed under the Apache License, Version 2.0 (the "License");
• you may not use this file except in compliance with the License.
• You may obtain a copy of the License at
• http://www.apache.org/licenses/LICENSE-2.0
• Unless required by applicable law or agreed to in writing, software
• distributed under the License is distributed on an "AS IS" BASIS,
• WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
• See the License for the specific language governing permissions and
• limitations under the License. */

package com.google.android.filament.gltf2

import android.annotation.SuppressLint import android.app.Activity import android.os.Bundle import android.util.Log import android.view. import com.google.android.filament.Engine import com.google.android.filament.Fence import com.google.android.filament.IndirectLight import com.google.android.filament.Scene import com.google.android.filament.Skybox import com.google.android.filament.View import com.google.android.filament.utils. import java.nio.ByteBuffer

class MainActivityMultiInstance : Activity() {

companion object {
    // Load the library for the utility layer, which in turn loads gltfio and the Filament core.
    init { Utils.init() }
    private const val TAG = "gltf-viewer"
}

private lateinit var surfaceView: SurfaceView
private lateinit var choreographer: Choreographer
private val frameScheduler = FrameCallback()
private lateinit var modelViewer: ModelViewer

private lateinit var surfaceView2: SurfaceView
private lateinit var choreographer2: Choreographer
private val frameScheduler2 = FrameCallback2()
private lateinit var modelViewer2: ModelViewer

@SuppressLint("ClickableViewAccessibility")
override fun onCreate(savedInstanceState: Bundle?) {
    super.onCreate(savedInstanceState)
    setContentView(R.layout.simple_layout_multi_instance)
    window.addFlags(WindowManager.LayoutParams.FLAG_KEEP_SCREEN_ON)

    surfaceView = findViewById(R.id.main_sv)
    choreographer = Choreographer.getInstance()
    modelViewer = ModelViewer(surfaceView)
    surfaceView.setOnTouchListener { _, event ->
        modelViewer.onTouchEvent(event)
        true
    }
    createDefaultRenderables(modelViewer)
    createIndirectLight(modelViewer)
    setViewOptions(modelViewer)

    surfaceView2 = findViewById(R.id.main_sv2)
    choreographer2 = Choreographer.getInstance()
    modelViewer2 = ModelViewer(surfaceView2)
    surfaceView2.setOnTouchListener { _, event ->
        modelViewer2.onTouchEvent(event)
        true
    }
    createDefaultRenderables(modelViewer2)
    createIndirectLight(modelViewer2)
    setViewOptions(modelViewer2)
}

private fun setViewOptions(modelViewer: ModelViewer){
    val view = modelViewer.view

    // on mobile, better use lower quality color buffer
    view.renderQuality = view.renderQuality.apply {
        hdrColorBuffer = View.QualityLevel.MEDIUM
    }

    // dynamic resolution often helps a lot
    view.dynamicResolutionOptions = view.dynamicResolutionOptions.apply {
        enabled = true
        quality = View.QualityLevel.MEDIUM
    }

    // MSAA is needed with dynamic resolution MEDIUM
    view.multiSampleAntiAliasingOptions = view.multiSampleAntiAliasingOptions.apply {
        enabled = true
    }

    // FXAA is pretty cheap and helps a lot
    view.antiAliasing = View.AntiAliasing.FXAA

    // ambient occlusion is the cheapest effect that adds a lot of quality
    view.ambientOcclusionOptions = view.ambientOcclusionOptions.apply {
        enabled = true
    }

    // bloom is pretty expensive but adds a fair amount of realism
    view.bloomOptions = view.bloomOptions.apply {
        enabled = true
    }
}

private fun createDefaultRenderables(modelViewer: ModelViewer) {
    val buffer = assets.open("models/scene.gltf").use { input ->
        val bytes = ByteArray(input.available())
        input.read(bytes)
        ByteBuffer.wrap(bytes)
    }

    modelViewer.loadModelGltfAsync(buffer) { uri -> readCompressedAsset("models/$uri") }
    updateRootTransform(modelViewer)
}

private fun createIndirectLight(modelViewer: ModelViewer) {
    val engine = modelViewer.engine
    val scene = modelViewer.scene
    val ibl = "default_env"
    readCompressedAsset("envs/$ibl/${ibl}_ibl.ktx").let {
        scene.indirectLight = KTX1Loader.createIndirectLight(engine, it)
        scene.indirectLight!!.intensity = 30_000.0f
    }
    readCompressedAsset("envs/$ibl/${ibl}_skybox.ktx").let {
        scene.skybox = KTX1Loader.createSkybox(engine, it)
    }
}

private fun readCompressedAsset(assetName: String): ByteBuffer {
    val input = assets.open(assetName)
    val bytes = ByteArray(input.available())
    input.read(bytes)
    return ByteBuffer.wrap(bytes)
}

override fun onResume() {
    super.onResume()
    choreographer.postFrameCallback(frameScheduler)
    choreographer2.postFrameCallback(frameScheduler2)
}

override fun onPause() {
    super.onPause()
    choreographer.removeFrameCallback(frameScheduler)
    choreographer2.removeFrameCallback(frameScheduler2)
}

override fun onDestroy() {
    super.onDestroy()
    choreographer.removeFrameCallback(frameScheduler)
    choreographer2.removeFrameCallback(frameScheduler2)
}

private fun updateRootTransform(modelViewer: ModelViewer) {
    modelViewer.transformToUnitCube()
}

inner class FrameCallback : Choreographer.FrameCallback {
    private val startTime = System.nanoTime()
    override fun doFrame(frameTimeNanos: Long) {
        choreographer.postFrameCallback(this)

        modelViewer.animator?.apply {
            if (animationCount > 0) {
                val elapsedTimeSeconds = (frameTimeNanos - startTime).toDouble() / 1_000_000_000
                applyAnimation(0, elapsedTimeSeconds.toFloat())
            }
            updateBoneMatrices()
        }

        modelViewer.render(frameTimeNanos)
    }
}

inner class FrameCallback2 : Choreographer.FrameCallback {
    private val startTime = System.nanoTime()
    override fun doFrame(frameTimeNanos: Long) {
        choreographer2.postFrameCallback(this)

        modelViewer2.animator?.apply {
            if (animationCount > 0) {
                val elapsedTimeSeconds = (frameTimeNanos - startTime).toDouble() / 1_000_000_000
                applyAnimation(0, elapsedTimeSeconds.toFloat())
            }
            updateBoneMatrices()
        }

        modelViewer2.render(frameTimeNanos)
    }
}

}

[romainguy]

It's a crash in the OpenGL driver, there's probably not much we can do about it. It seems the driver is crashing when starting one of its own threads.

[oceanii]

Thank you for your answer, I have two confusions:

1. If I cancel this code, it will return to normal, but I don't know the reason. It seems that enabled bloom is causing this problem view.bloomOptions = view.bloomOptions.apply { enabled = true } 2.Does the filament engine support multiple instances and has been tested and verified?

[romainguy]

Yes, you can use multiple instances of Filament (although it's expensive, it's better instead to create one Engine and share it across multiple Renderer). Again, the problem is the GL driver.

[oceanii]

thanks，I will continue to follow up

[pixelflinger]

Yes, you can use multiple instances of Filament (although it's expensive, it's better instead to create one Engine and share it across multiple Renderer).

Unless you need to use Filament from multiple threads.

[pixelflinger]

@oceanii could you please try with a debug build of Filament? We should get a better stack trace.

[oceanii]

Yes, you can use multiple instances of Filament (although it's expensive, it's better instead to create one Engine and share it across multiple Renderer).

Unless you need to use Filament from multiple threads.

I need to use two SurfaceView and two Engine on one activity，these two engine instances render simultaneously, one instance cannot reach the target.

[oceanii]

@oceanii could you please try with a debug build of Filament? We should get a better stack trace.

Thank you for your answer. I build and run from filament/android/Windows.md based on 1.38.0，If using debug version，which branch should I use and how to build

[romainguy]

You can use a single Engine to render into two SurfaceViews. It's better in terms of resource usage.

[oceanii]

You can use a single Engine to render into two SurfaceViews. It's better in terms of resource usage.

I plan to try a single Engine to render into two SurfaceViews, If there is only one engine and one render, can it achieve the effect of screenshot above? (Simultaneous rendering and appearance of two effects)

[pixelflinger]

You can use a single Engine to render into two SurfaceViews. It's better in terms of resource usage.

I plan to try a single Engine to render into two SurfaceViews, If there is only one engine and one render, can it achieve the effect of screenshot above? (Simultaneous rendering and appearance of two effects)

When you think about it, there is only a single GPU, using multiple engines is not going to change that. At the end of the day, rendering happens sequentially.

[romainguy]

You don't even need two SurfaceViews to do what's on your screenshot. You can use a single Engine, a single Renderer and multiple Filament Views (com.google.android.filament.View objects).

In the following screenshot, there's only 1 Engine and 1 Renderer but 4 Views:

Those views are showing the same scene from different cameras (and different post processing effects) but you can render multiple scenes as well. The UI on the left is another Filament View rendering a different scene.

[oceanii]

I understand this method now, thank you very much

[August1996]

I meet the same problem, and this issue seems become more serious from 1.31.2 to 1.39.0