Model does not unload from memory after removing from DOM

[Pterko]

Description:

A loaded model (gltf-model) doesn't unload from memory after it removed from DOM.

• A-Frame Version: 0.7.0
• Platform / Device: At least Windows/Chrome (possibly all platforms)
• Reproducible Code Snippet or URL: https://pterko.github.io/aframe/memory_bug_demo/test.html

How to reproduce:

• Wait until page loads
• When page loads, take a memory snapshot
• Wait 10-15s before model will appear and loaded, take a snapshot
• Wait until model will be deleted, take memory snapshot again

Expected behavior: amount of used memory decreases Real behavior: amount of used memory don't change

[calrk]

I've noticed this in my project as well, though I use a sllightly different version of the gltf loader. When I unmount/unload any gltf object, I run it through a function as below to make sure to dispose of any material/geometry/textures.

Could add this somewhere to the a-frame source but not sure where would be appropriate or if there is a simpler way of doing it.

export const disposeObject3D = object => {
  object.traverse(obj => {
    if(obj.material){
      obj.material.dispose();
      if(obj.material.map){
        obj.material.map.dispose();
      }
      if(obj.material.lightMap){
        obj.material.lightMap.dispose();
      }
      if(obj.material.aoMap){
        obj.material.aoMap.dispose();
      }
      if(obj.material.emissiveMap){
        obj.material.emissiveMap.dispose();
      }
      if(obj.material.bumpMap){
        obj.material.bumpMap.dispose();
      }
      if(obj.material.normalMap){
        obj.material.normalMap.dispose();
      }
      if(obj.material.displacementMap){
        obj.material.displacementMap.dispose();
      }
      if(obj.material.roughnessMap){
        obj.material.roughnessMap.dispose();
      }
      if(obj.material.metalnessMap){
        obj.material.metalnessMap.dispose();
      }
      if(obj.material.alphaMap){
        obj.material.alphaMap.dispose();
      }
    }
    if(obj.geometry){
      obj.geometry.dispose();
    }
  });
}

[donmccurdy]

@calrk Did your function above resolve the memory leaks for your project?

A-Frame uses THREE.Cache, so it's expected that caching will stick around until THREE.Cache.clear() is called. But otherwise we're not intentionally retaining things.

[calrk]

@donmccurdy Yeah this resolved memory leaks when swapping/removing gltf models often.

I'm not exactly sure how the dispose functions work, they emit the following: this.dispatchEvent( { type: 'dispose' } ); in ThreeJS which triggers events on the renderers to make sure the assets are fully cleared up and deallocated the resources to them.

Otherwise I believe that even if you clear your cache and your local variables, there is still references to those variables on the renderer which it keeps for performance optimisations until you clear them up.

[donmccurdy]

Makes sense, thanks. I don't think we want to have that function run automatically when an element is removed, because uploading textures to the GPU is expensive. For example, the user might just detaching it and moving it to another parent, and we don't want to force a texture to upload again.

Brainstorming ways to let users do this themselves:

• Create entity.dispose() method, that traverses entity.object3D.children... and disposes any textures and geometry that aren't in use elsewhere (can we detect that?)
• Create dispose-on-detach component that users can attach to entities, so that the entity will automatically dispose of textures when detached.

[calrk]

Hmm, just taking a look at the code. It looks like GLTFLoader uses GLTFRegistry not THREE.Cache like you mentioned. This means each model would load its own resources and they would be safe to remove, but means that the resources get loaded and allocated separately each time. So if a user detached the object and reattached it to another, all the resources would get reloaded using a different instance of the GLTFLoader/GLTFRegistry?

I thought that it was using THREE.Cache but maybe that is just the a-asset-item that gets cached, not the actual loaded geometries/materials? Correct me if I'm wrong...

I'm not sure of any existing way to check how many objects are referencing a resource. The only way I could think of doing it is traversing the whole scene graph and checking against UUIDs, but that might be a bit much? I think it should be disposed of automatically as its assumed that when a node is removed, the memory it takes up should be removed to but would leave that call to someone else.

[donmccurdy]

The GLTFRegistry cache is only retained while the model loads, so it shouldn't be related to this issue.

I thought that it was using THREE.Cache but maybe that is just the a-asset-item that gets cached, not the actual loaded geometries/materials? Correct me if I'm wrong...

The A-Frame asset system will cache whatever files it refers directly to. So if you have a .glb with textures and payload inside it, that's caching everything. If you have a .gltf with external payload and textures, most things are not using THREE.Cache.

[calrk]

I see... I mostly use .gltfs with separate binaries and textures, so that's why the code works fine for me. If you have multiple .glbs then yes it would cause issues if you disposed of resources that are in use elsewhere.

I can't think of a clean way that doesn't require a lot of changes. Maybe if it were something like THREE.Scene.Clear()/THREE.Renderer.Clear() that goes through all the resources and clears any that are not being used, but thats probably quite a big task. I think the dispose-on-detach is good but not an obvious solution for people to use, would an attribute on the a-gltf-model make more sense?

[donmccurdy]

There is WebGLRenderer.dispose() ... for cases where you want to clear the scene entirely, we should probably be using that in a AScene.detachedCallback. Would welcome PRs if someone has a use case and can test that.

I think the dispose-on-detach is good but not an obvious solution for people to use, would an attribute on the a-gltf-model make more sense?

I'd rather not do anything specific to a single format, because we'll just have to re-implement it for other formats. I'm (more) ok with non-obvious solutions, because there are likely to be non-obvious pitfalls to using it:

1. Tracking references to textures, materials, etc. could be hard. Unless we solve that, disposing has the potential to 'break' other things in the scene.
2. Uploading textures to the GPU can easily take 500ms for a 10mb model, that that's a huge number of dropped frames in a running VR scene. So, dispose-by-default might be risky if you're just trying to move the model from parentA to parentB.

I think I like the entity.dispose() method more than making a component (and we could always make a component that applies it for you).

[dmarcos]

The component API has a remove method that is invoked when an entity is detached from the DOM. Should not each component clean after itself when removed?

[donmccurdy]

If the remove method disposed of textures, then moving a child from one parent to another would re-upload textures to the GPU, and almost certainly drop frames. Hard to automatically guess what to do. ☹️

Could wait a few frames then check if entity is still detached?

[florentpeyrard]

Hi @donmccurdy , hi @dmarcos ,

having the same issue, very limitating for anything bigger than quite small games. As a user, I'm very interested in the entity.dispose() feature because I know when I need to clean the memory. Do you plan to implement it ?

[Pterko]

Hi, @florentpeyrard It's so bad to hear that issue is still here. Have you tried disposeObject3D, that was suggested above? It can work as a workaround for you. Also, we modified that function, so it should remove all things from memory more properly.

function freeObjectFromMemory(object, domReference) {
  object.traverse(function(obj){
      if (obj.material) {
        obj.material.dispose();
        if (obj.material.map) {
          obj.material.map.dispose();
        }
        if (obj.material.lightMap) {
          obj.material.lightMap.dispose();
        }
        if (obj.material.aoMap) {
          obj.material.aoMap.dispose();
        }
        if (obj.material.emissiveMap) {
          obj.material.emissiveMap.dispose();
        }
        if (obj.material.bumpMap) {
          obj.material.bumpMap.dispose();
        }
        if (obj.material.normalMap) {
          obj.material.normalMap.dispose();
        }
        if (obj.material.displacementMap) {
          obj.material.displacementMap.dispose();
        }
        if (obj.material.roughnessMap) {
          obj.material.roughnessMap.dispose();
        }
        if (obj.material.metalnessMap) {
          obj.material.metalnessMap.dispose();
        }
        if (obj.material.alphaMap) {
          obj.material.alphaMap.dispose();
        }
      }
      if (obj.geometry) {
        obj.geometry.dispose();
        obj.geometry.attributes.color = {};
        obj.geometry.attributes.normal = {};
        obj.geometry.attributes.position = {};
        obj.geometry.attributes.uv = {};
        obj.geometry.attributes = {};
        obj.material = {};
      }
  })

  for (var elem in THREE.Cache.files) {
    if (elem.startsWith('./assets/items/')) {
      THREE.Cache.files[elem] = "";
      THREE.Cache.remove(elem);
    }
  }

  if (domReference.object3DMap.mesh) {
    domReference.object3DMap.mesh.materialLibraries = {};

    var start_elem = domReference.object3DMap.mesh;
    var elem = start_elem;
    // while (true) {
    //   if (elem.children.length < 2) {
    //     elem = elem.children[0];
    //   } else {
    //     break;
    //   }
    // }

    var elems = elem.children;
    for (var el of elems) {
      for (var key in el) {
        el[key] = {};
      }
    }
  }
}

We used this function like this:

  var objectsToDelete = [];
  var items = document.querySelectorAll('.item');
  for (var iIndex = 0; iIndex < items.length; iIndex++){
    objectsToDelete.push(items[iIndex]);
  }
  var scene = document.querySelector('#scene');
  objectsToDelete.push(scene);
  for (var i = 0; i < objectsToDelete.length; i++) {
    freeObjectFromMemory(objectsToDelete[i].object3D, objectsToDelete[i]);    
  }

This function not only destroys all maps and attributes of given entities but also removes all cache files of Three.js from memory (this will be useful if you're loading gltf models or png atlases from url). You will need to change ./assets/items/ in the code to your directory, where assets are stored. This is not very user-friendly, but it should work. We've implemented this function for our aframe experience, that was split into rooms with 50-80MB of objects and it worked.

[florentpeyrard]

Hi @Pterko , thank you very much for the detailed explanations and the quick reply ! I struggled a lot but finally achieved some improvements to managing memory in my app, but I can't get a perfect result : tracking sceneElement.renderer.info, I can see there are still some textures which accumulate. Same for geometries, by the way.

Here is my code, for the record :

    disposeObject3D : function (object) {

        object.traverse(obj => {

            if (obj instanceof THREE.Mesh) {

                if(obj.material){

                    obj.material.dispose()
                    if(obj.material.map){
                        obj.material.map.dispose()
                        obj.material.map = null
                    }
                    if(obj.material.lightMap){
                        obj.material.lightMap.dispose()
                        obj.material.lightMap = null
                    }
                    if(obj.material.aoMap){
                        obj.material.aoMap.dispose()
                        obj.material.aoMap = null
                    }
                    if(obj.material.emissiveMap){
                        obj.material.emissiveMap.dispose()
                        obj.material.emissiveMap = null
                    }
                    if(obj.material.bumpMap){
                        obj.material.bumpMap.dispose()
                        obj.material.bumpMap = null
                    }
                    if(obj.material.normalMap){
                        obj.material.normalMap.dispose()
                        obj.material.normalMap = null
                    }
                    if(obj.material.displacementMap){
                        obj.material.displacementMap.dispose()
                        obj.material.displacementMap = null
                    }
                    if(obj.material.roughnessMap){
                        obj.material.roughnessMap.dispose()
                        obj.material.roughnessMap = null
                    }
                    if(obj.material.metalnessMap){
                        obj.material.metalnessMap.dispose()
                        obj.material.metalnessMap = null
                    }
                    if(obj.material.alphaMap){
                        obj.material.alphaMap.dispose()
                        obj.material.alphaMap = null
                    }
                    if(obj.material.envMaps){
                        obj.material.envMaps.dispose()
                        obj.material.envMaps = null
                    }
                    if(obj.material.envMap){
                        obj.material.envMap.dispose()
                        obj.material.envMap = null
                    }
                    if(obj.material.specularMap){
                        obj.material.specularMap.dispose()
                        obj.material.specularMap = null
                    }
                    if(obj.material.gradientMap){
                        obj.material.gradientMap.dispose()
                        obj.material.gradientMap = null
                    }

                }
                if(obj.geometry){
                    obj.geometry.dispose()
                }
                if(obj.texture){
                    obj.texture.dispose()
                    obj.texture = {}
                }
                if(obj.bufferGeometry){
                    obj.bufferGeometry.dispose()
                }

            }
         })

    }   

I run disposeObject3D on the concerned entities' object3D, then I destroy the entities (in a data-driven way, as I use vue.js). Removing fields in object3Ds was leading to errors and I didn't see benefits, so I kept them. I prefered to keep the cache as well, as models may be needed later in my case.

I believe such a function should be either called by the remove method or at least be available as a helper function and dynamic loading should be documented. All of this would make sense only when we are sure of how memory is managed by three.js. Once again, I observed some sticking textures (and geometries) and couldn't figure out what they are (I tracked materials textures with three.js material.toJSON() which I ran on every concerned scene object after I have cleaned the scene, and didn't find any remaining texture. Maybe some bug in three.js).

For now, dynamic unloading is very tricky, requires to deep dive into three.js, and problems are likely to happen if the experience/game you build has a large amount of assets. I think I'll monitor memory textures amount and refresh the page when it gets too high.

[IntraNoctem]

Hi,

unfortunately we are facing the increasing memory usage problem too. In our case we have a-frame in a popup component similar to the sketchfab view. Therefore a-frame gets rendered and removed from the dom several times.

I've created a tiny fiddle with a scene that is rendered and removed 10 times and contains only a blue sky: https://jsfiddle.net/nwz6okm0/

We already tried the dispose method, but the memory usage increases anyway. Inspecting the browser's memory usage while watching the fiddle we have an average increase of 30 MB each time the scene is rendered. The increase is even higher in our more complex application.

Does anyone have an idea how to solve the problem? Or are we just using the dispose method incorrectly?

Thanks :)

[Dirk-27]

Hi,

you see that the memory increases also with the default camera and without the sky. You can see that only in chrome (Version 92.0.4515.131 (64-Bit)) and not with firefox.

So there seems to be a bug related to inserting a new a-scene and chrome.

[Dirk-27]

I opened #4899 for the allocation of memory in chrome per scene.

[shi11]

Hi, what ultimately worked for my memory leak, besides properly disposing of the model was to call

renderer.info.reset()

https://threejs.org/docs/#api/en/renderers/WebGLRenderer

[arpu]

Should this be Reporter to threejs?

[vincentfretin]

I didn't see this issue existed when I wrote a few days ago this page https://aframe.wiki/en/memory to explain why you have a memory leak when loading and removing a glb with gltf-model component or removing the scene, which still apply with aframe 1.3.0. The fix I describe is specific to the gltf-model component though, not a generic solution for all components that load a model.