VC4 kernel framework driver - memory leak

[nkichukov]

Hi all, (redirected here via: https://github.com/raspberrypi/linux/issues/2293)

I have identified a memory leak in the VC4 kernel framework driver implementation on raspberry pi 3b. Details follow below:

kernel: 4.14.0-v7+ compiler: gcc-6.4 raspberry pi running in 32bit mode OS: Gentoo Linux Kernel configuration is attached, see below.

Whenever kodi(v17.6) is playing video, I can see that /proc/slabinfo reports number of kmalloc-128 constantly increasing and never releasing: kmalloc-128 84916 84924 384 21 2 : tunables 0 0 0 : slabdata 4044 4044 0

In about 24 hours this has taken up about 600MB of the memory and the Pi needs to be restarted to avoid swapping. If I stop kodi playback and leave it idle, the kmalloc-128 remains the same until playback is resumed which starts increasing again. If I stop the X server and kodi altogether, the memory is not released.

The memleak pattern is like the one below

(from /sys/kernel/debug/kmemleak):

unreferenced object 0x95ce16c0 (size 128): comm "kodi.bin", pid 2075, jiffies 891363 (age 2338.023s) hex dump (first 32 bytes): 01 00 00 00 5c ed 1e 7f 00 00 00 00 00 00 00 00 ............... d0 16 ce 95 d0 16 ce 95 00 3a fb b4 00 00 00 00 .........:...... backtrace: [<802827c8>] kmem_cache_alloc_trace+0x234/0x2f8 [<7f1e461c>] vc4_submit_cl_ioctl+0x730/0xb50 [vc4] [<7f0d796c>] drm_ioctl_kernel+0x78/0xb8 [drm] [<7f0d7cd4>] drm_ioctl+0x1b4/0x37c [drm] [<802ac090>] do_vfs_ioctl+0xb0/0x8b4 [<802ac8d8>] SyS_ioctl+0x44/0x6c [<80108140>] ret_fast_syscall+0x0/0x28 [] 0xffffffff

Let me know if additional information is required to track this issue down.

rpi-4.14.y-kernel_configuration_file.gz

Saw another issue still open here for memory leak, but what I report happens with pretty high CMA, 256MB, so not related to low CMA setting. Also the trace is different.

Thank you, -Nikolay

[stschake]

I think it's leaking the dma_fences, certainly the "01" refcount in the hexdump looks suspiciously like them. Could you try the attached patch?

0001-drm-vc4-Release-fence-after-signalling.txt

(Sorry, filename is off because GitHub, ridiculously so, does not allow attaching .patch files)

[nkichukov]

Hello Stefan, thank you for the quick response! It fixed the huge memleak issue! Thank you very much!

However, another memleak appeared(it may already have been there before, I just did not see it), see below:

unreferenced object 0x842aa340 (size 128): comm "X", pid 2009, jiffies 7034334 (age 301.527s) hex dump (first 32 bytes): 50 3f eb b6 01 00 00 00 00 00 00 00 00 00 00 00 P?.............. 50 a3 2a 84 50 a3 2a 84 00 00 00 00 00 00 00 00 P..P.......... backtrace: [<802827c8>] kmem_cache_alloc_trace+0x234/0x2f8 [<7f167ae8>] drm_atomic_helper_setup_commit+0x1cc/0x3b0 [drm_kms_helper] [<7f1e2aa0>] vc4_atomic_commit+0x30/0x130 [vc4] [<7f0f44a0>] drm_atomic_commit+0x5c/0x68 [drm] [<7f0f5700>] drm_atomic_connector_commit_dpms+0xf8/0x108 [drm] [<7f0fae64>] drm_mode_obj_set_property_ioctl+0x1bc/0x2b4 [drm] [<7f0f9820>] drm_mode_connector_property_set_ioctl+0x48/0x50 [drm] [<7f0e296c>] drm_ioctl_kernel+0x78/0xb8 [drm] [<7f0e2cd4>] drm_ioctl+0x1b4/0x37c [drm] [<802ac090>] do_vfs_ioctl+0xb0/0x8b4 [<802ac8d8>] SyS_ioctl+0x44/0x6c [<80108140>] ret_fast_syscall+0x0/0x28 [] 0xffffffff unreferenced object 0x83ee2340 (size 128): comm "X", pid 2009, jiffies 7044631 (age 291.296s) hex dump (first 32 bytes): 50 3f eb b6 01 00 00 00 00 00 00 00 00 00 00 00 P?.............. 50 23 ee 83 50 23 ee 83 00 00 00 00 00 00 00 00 P#..P#.......... backtrace: [<802827c8>] kmem_cache_alloc_trace+0x234/0x2f8 [<7f167ae8>] drm_atomic_helper_setup_commit+0x1cc/0x3b0 [drm_kms_helper] [<7f1e2aa0>] vc4_atomic_commit+0x30/0x130 [vc4] [<7f0f44a0>] drm_atomic_commit+0x5c/0x68 [drm] [<7f0f5700>] drm_atomic_connector_commit_dpms+0xf8/0x108 [drm] [<7f0fae64>] drm_mode_obj_set_property_ioctl+0x1bc/0x2b4 [drm] [<7f0f9820>] drm_mode_connector_property_set_ioctl+0x48/0x50 [drm] [<7f0e296c>] drm_ioctl_kernel+0x78/0xb8 [drm] [<7f0e2cd4>] drm_ioctl+0x1b4/0x37c [drm] [<802ac090>] do_vfs_ioctl+0xb0/0x8b4 [<802ac8d8>] SyS_ioctl+0x44/0x6c [<80108140>] ret_fast_syscall+0x0/0x28 [] 0xffffffff unreferenced object 0x84bbe040 (size 128): comm "X", pid 2009, jiffies 7049751 (age 286.242s) hex dump (first 32 bytes): 50 3f eb b6 01 00 00 00 00 00 00 00 00 00 00 00 P?.............. 50 e0 bb 84 50 e0 bb 84 00 00 00 00 00 00 00 00 P...P........... backtrace: [<802827c8>] kmem_cache_alloc_trace+0x234/0x2f8 [<7f167ae8>] drm_atomic_helper_setup_commit+0x1cc/0x3b0 [drm_kms_helper] [<7f1e2aa0>] vc4_atomic_commit+0x30/0x130 [vc4] [<7f0f44a0>] drm_atomic_commit+0x5c/0x68 [drm] [<7f0f5700>] drm_atomic_connector_commit_dpms+0xf8/0x108 [drm] [<7f0fae64>] drm_mode_obj_set_property_ioctl+0x1bc/0x2b4 [drm] [<7f0f9820>] drm_mode_connector_property_set_ioctl+0x48/0x50 [drm] [<7f0e296c>] drm_ioctl_kernel+0x78/0xb8 [drm] [<7f0e2cd4>] drm_ioctl+0x1b4/0x37c [drm] [<802ac090>] do_vfs_ioctl+0xb0/0x8b4 [<802ac8d8>] SyS_ioctl+0x44/0x6c [<80108140>] ret_fast_syscall+0x0/0x28 [] 0xffffffff

Thank you, -Nik

[stschake]

That is an unrelated leak (another refcounting fail, 01 00..)

[nkichukov]

Hello Stefan, If this is unrelated to the vc4 kernel driver, do I report this to the raspberrypi kernel developers? Or (if vc4 related) would you like to have a separate issue for this leak here(anholt) and track separately so this can be closed and the patch merged?

Thank you, -Nik

[lategoodbye]

Since this is also vc4 related, it should be reported here in this repository and not to the raspberrypi guys.

[nkichukov]

Thank you Stefan. The originally reported issue was resolved by the provided patch. I am closing the case and will open a new one for the second memory leak.