search

google / filament

Filament is a real-time physically based rendering engine for Android, iOS, Windows, Linux, macOS, and WebGL2
https://google.github.io/filament/
Apache License 2.0
17.8k stars 1.89k forks source link

Vulkan GPU Memory Leak and Crash #7579

Closed Nolram12345 closed 8 months ago

Nolram12345 commented 8 months ago

⚠️ Issues not using this template will be systematically closed.

Describe the bug On Windows, when using Vulkan, on some GLTF scenes GPU memory usage increases constantly during execution, leading the application to crash due to running out of memory.

To Reproduce Steps to reproduce the behavior:

  1. Download the following scene as GLB file from sketchfab: Test Asset by Pixel
  2. Launch gltf_viewer.exe
  3. Observe the application run out of memory and crash.

Expected behavior The application memory should stay constant when no new data is created, and should not run out of memory and crash.

Screenshots As seen in Task Manager: image

Desktop (please complete the following information):

Nolram12345 commented 8 months ago

In my brief testing, this issue does not seem to happen on the OpenGL backend.

Nolram12345 commented 8 months ago

Any updates on this? Has this issue been reproduced?

Nolram12345 commented 8 months ago

I'm still able to reproduce this issue in v1.50.4 - any updates?

poweifeng commented 8 months ago

Sorry for the delay. I'll try to repro this week. It's likely that I'll address this in an upcoming refactor, but it might take a little while more.

jeanlemotan commented 8 months ago

This might not be the case anymore after the Vulkan refactor, but last time I checked the pipeline cache, it was using the render target pointer or texture as a key, and if recreating the render target per frame the cache will invalidate per frame, causing it to grow a lot. I think a better option would be to use the RT/Texture descriptors (formats) instead as key, not the pointers directly. This is relevant when RT/SwapChains are big, like on desktop or high res devices, too big to fit in the default filament pools.

Nolram12345 commented 8 months ago

Is there any ETA for a fix for this problem? It's proving to be a massive issue in production.

poweifeng commented 8 months ago

Still in progress. No hard ETA unfortunately, but I expect some result in these two weeks.

Nolram12345 commented 8 months ago

This seems be resolved in the latest version - at least on my hardware! Might be worth looking into further, but I think I can close this issue for now.

poweifeng commented 8 months ago

hmm, interesting. So the cause might be this: https://github.com/google/filament/commit/d05c61fe9a982035caecfea179896b4111fb28f3

Thanks for the update!

Nolram12345 commented 8 months ago

Might have been! It's definitely worth further testing. I'll keep my eye on future releases and re-open the issue if it comes up again. Testing is as simple as opening the GLTF viewer with something loaded and keeping it running for a few minutes :)