[Critical] [Bug] Memory leaks [SOLUTION INCLUDED] [FIXED]

[kustraslawomir]

@ThomasGorisse Sorry to worry You, but I have a huge problem with memory.

I'm trying everything, but nothing works.

    private fun reclaimReleasedResources() {
        SceneView.reclaimReleasedResources()
        Renderer.reclaimReleasedResources()
        ResourceManager
            .getInstance()
            .reclaimReleasedResources()
        ResourceManager
            .getInstance()
            .destroyAllResources()
    }

 arSceneView?.apply {
            scene?.callOnHierarchy { node ->
                try {
                    node.parent = null
                    if (node is AnchorNode) {
                        TimberLog.e("[CLEAN_UP] Detach: $node")
                        node.renderableInstance?.clear()
                        node.anchor?.apply {
                            detach()
                        }
                    }
                } catch (e: UnsupportedOperationException) {
                    TimberLog.e("[CLEAN_UP] Error: ${e.message}")
                }
            }
            destroySession()
        }

        arCharactersViewModel.getArCharacters().forEach { character ->
            character.apply {
                animationController.clear()
                cloud.node.renderableInstance.clear()
                distance.node.renderableInstance.clear()
                arCharacterNode.renderableInstance.clear()
            }
        }

        EngineInstance
            .getEngine()
            .filamentEngine
            .flushAndWait()

        locationArScene.clearRouteSteps()
        locationArScene.clearArObjects()
        locationArScene.stopLocationScene()

fun RenderableInstance.clear() {
    filamentAsset?.releaseSourceData()
    detachFromRenderer()
    destroy()
}

What else can I do? I can't migrate to sceneview as my project is too large at this point.

Each session (for example navigation to arFragment, add +- 100mb to memory stack. So after 10-15 sessions app crashes due to memory usage. Is anything I can do?

Some code with load AR models:

 try {
            getModelRenderable(
                path,
                context
            )?.thenAccept { renderable ->

                interaction.locations.forEach { location ->
                    viewModelScope.launch(Dispatchers.Main) {

                        val touch = ViewRenderable.builder()
                            .setView(context.get(), R.layout.view_character_touch)
                            .setAsyncLoadEnabled(true)
                            .await()
                            .apply {
                                isShadowCaster = false
                                isShadowReceiver = false
                            }

                        val interactionView = ViewRenderable.builder()
                            .setView(context.get(), R.layout.view_character_interaction)
                            .setAsyncLoadEnabled(true)
                            .await()
                            .apply {
                                isShadowCaster = false
                                isShadowReceiver = false
                            }

                        val distance = ViewRenderable.builder()
                            .setView(context.get(), R.layout.view_distance)
                            .setAsyncLoadEnabled(true)
                            .await()
                            .apply {
                                isShadowCaster = false
                                isShadowReceiver = false
                            }

                        val cloud = ViewRenderable.builder()
                            .setView(context.get(), R.layout.navigation_cloud)
                            .setAsyncLoadEnabled(true)
                            .await()
                            .apply {
                                isShadowCaster = false
                                isShadowReceiver = false
                            }

                        val transformableNode = TransformableNode(transformation).apply {
                            this.renderable = renderable
                        }

                        val nodeCloud = getNode(cloud)
                        val distanceCloud = getNode(distance)
                        transformableNode.renderableInstance.setCulling(false)

                        _arCharacterObjects.add(
                            ArCharacter(
                                renderState = RenderState.NONE,
                                width = interaction.arObject.metadata.width,
                                height = interaction.arObject.metadata.height,
                                depth = interaction.arObject.metadata.depth,
                                arObjectCourse = interaction.arObject.arObjectCourse,
                                characterId = RandomString.invoke(),
                                interactionEntity = interaction,
                                viewAboveCharacter = interactionView,
                                touch = touch,
                                location = location,
                                interactionStarted = false,
                                arCharacterNode = transformableNode,
                                animationController = AnimationStateController(
                                    renderableInstance = transformableNode.renderableInstance
                                ),
                                name = interaction.name,
                                description = interaction.description,
                                manualPlacementAllowed = interaction.manualPlacementAllowed,
                                manualPlacementRadius = interaction.manualPlacementRadius,
                                modelVisibilityRadius = interaction.modelVisibilityRadius,
                                cloud = buildArObject(
                                    location.lat,
                                    location.long,
                                    1.5f,
                                    nodeCloud,
                                    NavigationArObject.RotationMode.FACE_USER,
                                    NavigationArObject.ScalingMode.GRADUAL,
                                    objectType = ObjectType.Cloud
                                ),
                                distance = buildArObject(
                                    location.lat,
                                    location.long,
                                    1.5f,
                                    distanceCloud,
                                    NavigationArObject.RotationMode.FACE_USER,
                                    NavigationArObject.ScalingMode.GRADUAL,
                                    objectType = ObjectType.Cloud
                                )
                            )
                        )
                        TimberLog.d("[PERFORMANCE_LOCATION_MARKER] Added character.")
                    }
                }
            }
            TimberLog.d("[PERFORMANCE_LOCATION_MARKER] Finished creating objects. Size: ${_arCharacterObjects.size}")
        } catch (e: Exception) {
            TimberLog.e("[LOCATION_MARKER] Error while loading character: $e")
        }

[kustraslawomir]

[kustraslawomir]

@ThomasGorisse

each memory jump is a session creation and closing

[kustraslawomir]

@ThomasGorisse without building ViewRenderable & ModelRenderable there are no leaks at all. Maybe +- 200MB, but not that much. The app is stable. How can I release renderable's? What I'm missing? :( Could You please take a chance to help with this?

[kustraslawomir]

on low-quality modelRenderable (1,5 MB) There is no big deal, only on larger models (like 10-15MB).

I guess, 1MB of .glb files leaves 8-10MB of locked RAM memory on each session.

[kustraslawomir]

Same Issue on the official sample.

[kustraslawomir]

halloween.glb

[kustraslawomir]

@ThomasGorisse https://github.com/google/filament/releases/tag/v1.22.2 engine: Fix some memory leaks.

Maybe it is fixed in filament 1.22.2?

[eqgis]

@kustraslawomir hello, Look at RenderableInstance#createFilamentAssetModelInstance(). Free up FilamentAssets. like this, use RenderableInstance#destroyGltfAsset()

public class RenderableInstance implements AnimatableModel {

    //desc added by ikkyu will be used when loading gltf resources.
    private AssetLoader loader;

  ...
    void createFilamentAssetModelInstance() {
        if (renderable.getRenderableData() instanceof RenderableInternalFilamentAssetData) {
            RenderableInternalFilamentAssetData renderableData =
                    (RenderableInternalFilamentAssetData) renderable.getRenderableData();

            Engine engine = EngineInstance.getEngine().getFilamentEngine();

            //updated by ikkyu
            loader =
                    new AssetLoader(
                            engine,
                            RenderableInternalFilamentAssetData.getMaterialProvider(),
                            EntityManager.get());

            FilamentAsset createdAsset = renderableData.isGltfBinary ? loader.createAssetFromBinary(renderableData.gltfByteBuffer)
                    : loader.createAssetFromJson(renderableData.gltfByteBuffer);
            ...
            }

  ...
    public void destroyGltfAsset(){
        if (loader == null)return;
        if (filamentAsset != null){
            loader.destroyAsset(filamentAsset);
            filamentAsset = null;
        }
    }

[kustraslawomir]

@eqgis What sceneform version do you use? I have a slightly different implementation. There is no destroyGltfAsset method or class member 'loader' variable.

Thank you, I will try to implement this.

[eqgis]

@kustraslawomir The earlier version I used. The method of destroyGltfAsset was I added. You can refer it.

[kustraslawomir]

@eqgis In sceneView/sceneform-android there is com.gorisse.thomas.sceneform\core\1.21.0

[kustraslawomir]

@eqgis ok thank you! <3

[eqgis]

@kustraslawomir You're welcome, study together.

[kustraslawomir]

@eqgis

I think with your solution memory usage is more optimal. Thank you!!!

[kustraslawomir]

@eqgis On the above screen I reopened the session with 12MB .glb file 10 times.

[kustraslawomir]

Full solution:

package com.ultron.ar.utils

import com.google.android.filament.Engine
import com.google.android.filament.EntityManager
import com.google.android.filament.gltfio.AssetLoader
import com.google.android.filament.gltfio.FilamentAsset
import com.google.android.filament.gltfio.UbershaderLoader
import com.google.ar.sceneform.rendering.EngineInstance
import com.google.ar.sceneform.rendering.RenderableInstance

object CleanAssetsUseCase {

    private var engine: Engine = EngineInstance.getEngine().filamentEngine
    private val materialProvider = UbershaderLoader(engine)

    private var loader = AssetLoader(
        engine,
        materialProvider,
        EntityManager.get()
    )

    fun destroyGltfAsset(filamentAsset: FilamentAsset?) {
        if (filamentAsset != null) {
            loader.destroyAsset(filamentAsset)
        }
    }
}

fun RenderableInstance?.clear() {
    this?.let { renderableInstance ->
        CleanAssetsUseCase
            .destroyGltfAsset(renderableInstance.filamentAsset)
        renderableInstance.detachFromRenderer()
        renderableInstance.destroy()
    }
}

@eqgis there might be some problems with CleanAssetsUseCase? What do You think?