anholt / linux

Other
134 stars 24 forks source link

Possible memory leak introduced in 5.10.47/5.10.48 #156

Open kwirk opened 3 years ago

kwirk commented 3 years ago

After upgrading from 5.10.46 to 5.10.48, I had issues with oom-killer killing Kodi in about 20mins of starting to watch a video. The system then became unstable, with other processes being randomly killed, and a system reboot was required.

Also tested 5.10.50 and 5.10.52 which still have this issue.

Issue appears to be related to kmalloc-128 constantly growing in size:

$ sudo slaptop -s c
Active / Total Objects (% used)    : 266001 / 289256 (92.0%)
 Active / Total Slabs (% used)      : 13992 / 13992 (100.0%)
 Active / Total Caches (% used)     : 137 / 265 (51.7%)
 Active / Total Size (% used)       : 93490.38K / 103413.99K (90.4%)
 Minimum / Average / Maximum Object : 0.17K / 0.36K / 16.00K

  OBJS ACTIVE  USE OBJ SIZE  SLABS OBJ/SLAB CACHE SIZE NAME                   
 83628  83586  99%    0.38K   3987       21     31896K kmalloc-128
 15316  14367  93%    0.55K    547       28      8752K inode_cache
 30720  30691  99%    0.27K   1024       30      8192K ext4_groupinfo_4k
 31424  28348  90%    0.25K   1964       16      7856K kmalloc-64
 24975  17937  71%    0.29K    925       27      7400K dentry
 26192  26153  99%    0.24K   1637       16      6548K kernfs_node_cache
 11169  11152  99%    0.45K    657       17      5256K radix_tree_node
  3728   1177  31%    0.95K    233       16      3728K ext4_inode_cache
 11222  10974  97%    0.26K    362       31      2896K vm_area_struct
  1168   1127  96%    2.00K     73       16      2336K kmalloc-1k
  1872   1779  95%    1.00K    117       16      1872K kmalloc-512
   392    378  96%    3.94K     49        8      1568K task_struct

Noted high number of drm_atomic_helper_setup_commit:

$ sudo sort -rn /sys/kernel/slab/kmalloc-128/alloc_calls | head
  80462 drm_atomic_helper_setup_commit+0x21c/0x7bc [drm_kms_helper] age=3/84268/167031 pid=276-602 cpus=0-3
    153 drm_property_create+0xa0/0x190 [drm] age=167039/167040/167044 pid=276 cpus=1
     93 snd_info_create_entry+0x28/0xe0 [snd] age=167039/167148/167184 pid=244-276 cpus=1
     47 drm_mode_duplicate+0x20/0x4c [drm] age=167032/167032/167032 pid=276 cpus=1
     28 drm_universal_plane_init+0x118/0x604 [drm] age=167039/167039/167039 pid=276 cpus=1
     17 __ipv6_dev_mc_inc+0x154/0x3c4 [ipv6] age=165848/166512/167270 pid=1-594 cpus=0-3
     14 v3d_fence_create+0x28/0x9c [v3d] age=3768/64040/165436 pid=322-323 cpus=0-2
     11 devm_snd_soc_register_component+0x2c/0x80 [snd_soc_core] age=167040/167054/167107 pid=261-276 cpus=1-2
     10 snd_ctl_new+0x34/0x8c [snd] age=167039/167068/167182 pid=244-276 cpus=1
      9 __drmm_add_action_or_reset+0x14/0x30 [drm] age=167038/167047/167078 pid=271-276 cpus=1

I downgraded to 5.10.46 and this has resolved the issue:

$ sudo grep drm_atomic_helper_setup_commit /sys/kernel/slab/kmalloc-128/alloc_calls
      9 drm_atomic_helper_setup_commit+0x214/0x790 [drm_kms_helper] age=1/236240/305496 pid=113-590 cpus=0-3
      1 drm_atomic_helper_setup_commit+0x538/0x790 [drm_kms_helper] age=1 pid=590 cpus=1
      1 drm_atomic_helper_setup_commit+0x6f4/0x790 [drm_kms_helper] age=304206 pid=590 cpus=0

My post on Raspberry Pi fourms for more background: https://www.raspberrypi.org/forums/viewtopic.php?f=53&t=316394