Network process use >100% CPU on Youtube UI navigation PI2 (Yocto)

[emutavchi]

Start youtube and try rapidly change video categories (Recommended, Trending, Music, etc)

export XDG_RUNTIME_DIR=/tmp
export WAYLAND_DISPLAY=main0

westeros --renderer libwesteros_render_dispmanx.so.0.0.0 --framerate 60 --display ${WAYLAND_DISPLAY} --enableCursor &

WPELauncher https://youtube.com/tv

See CPU usage in top

  PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND                                                                                                                                        
 1130 root      20   0  234452  30112  17416 S 172.8  4.9   1:10.01 WPENetworkProce                                                                                                                                
 1132 root      20   0  417736 144640  43420 R 111.9 23.7   1:23.16 WPEWebProcess 

Instructions on how to build image: https://github.com/Comcast/rdk-on-raspberrypi

[zdobersek]

I noticed this before when loading washingtonpost.com. At that time, the issue was CPU-costly decryption of the encrypted data. I suspect the same thing here.

The problem is most likely in the underlying networking and encryption libraries. This could be either a bug causing repetitive decryption, or a misconfiguration in some library that disables hardware-accelerated decryption.

[zak211]

I was not able to reproduce this issue using the yocto build on rpi3 The highest CPU% I could see was 35%

I used right and left keys on the keyboard to rapidly change the categories in Youtube as mentioned in description Here is the top usage from my test

  PID  PPID USER     STAT   VSZ %VSZ %CPU COMMAND
  505   496 root     S     228m  38%  35% /usr/bin/WPENetworkProcess 11
  508   496 root     S     413m  69%  17% /usr/bin/WPEWebProcess 15 -1
  496   423 root     S     209m  35%   1% WPELauncher https://youtube.com/tv
  436   423 root     S    58356  10%   1% westeros --renderer libwesteros_render_dispmanx.so.0.0.0 --framerate 60 --display main0 --enableCursor
  329     1 root     S     346m  58%   0% /usr/bin/webbridge -b

[emutavchi]

@zak211 I was using top from procps package, can you please try it.

[zak211]

hi @emutavchi I was able to rebuild using procps and do see the issue here is the top CPU usage from my test

top - 23:34:14 up 3 min,  2 users,  load average: 0.81, 0.22, 0.07
Tasks: 118 total,   2 running, 116 sleeping,   0 stopped,   0 zombie
%Cpu(s): 55.0 us,  5.3 sy,  0.0 ni, 38.7 id,  0.0 wa,  0.0 hi,  1.0 si,  0.0 st
KiB Mem :   608552 total,   384448 free,   144432 used,    79672 buff/cache
KiB Swap:        0 total,        0 free,        0 used.   452072 avail Mem 

  PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND                                                                                                                                      
  456 root      20   0  232348  29412  17224 S 155.9  4.8   0:23.81 WPENetworkProce                                                                                                                              
  459 root      20   0  392060 130112  40648 R  77.3 21.4   0:34.20 WPEWebProcess                                                                                                                                
  447 root      20   0  214968  20560  16784 S   3.9  3.4   0:00.99 WPELauncher                                                                                                                                  
  439 root      20   0   58356   2964   2616 S   2.3  0.5   0:00.71 westeros                                                                                                                                     
  328 root      20   0  354440  14932  12484 S   1.3  2.5   0:01.49 webbridge 

[zak211]

I tried the same on our buildroot rpi build without westeros and I still see pretty high CPU usage for WPENetworkProcess Here is the top output on rpi2 buildroot minus westeros

Tasks:  62 total,   3 running,  59 sleeping,   0 stopped,   0 zombie
%Cpu0  :  25.6/1.9    28[||||||||||||||||||||||||||||                                                                        ]
%Cpu1  :  35.9/4.5    40[||||||||||||||||||||||||||||||||||||||||                                                            ]
%Cpu2  :  31.3/2.0    33[|||||||||||||||||||||||||||||||||                                                                   ]
%Cpu3  :  32.9/0.0    33[|||||||||||||||||||||||||||||||||                                                                   ]
GiB Mem : 16.6/0.722    [                                                                                                    ]
GiB Swap:  0.0/0.000    [                                                                                                    ]

  PID USER      PR  NI    VIRT    RES  %CPU %MEM     TIME+ S COMMAND                                                                                                                                                                                                               
    1 root      20   0    1.8m   0.2m   0.0  0.0   0:04.26 S init                                                                                                                                                                                                                  
   92 root      20   0    1.8m   0.2m   0.0  0.0   0:00.00 S  `- syslogd                                                                                                                                                                                                           
   95 root      20   0    1.8m   0.2m   0.0  0.0   0:00.00 S  `- klogd                                                                                                                                                                                                             
   97 root      20   0    2.5m   1.6m   0.0  0.2   0:00.05 S  `- udevd                                                                                                                                                                                                             
  252 root      20   0    1.8m   0.1m   0.0  0.0   0:00.00 S  `- udhcpc                                                                                                                                                                                                            
  268 root      20   0    1.8m   0.1m   0.0  0.0   0:00.00 S  `- dropbear                                                                                                                                                                                                          
  353 root      20   0    2.1m   1.2m   0.0  0.2   0:00.27 S      `- dropbear                                                                                                                                                                                                      
  354 root      20   0    1.9m   1.5m   0.0  0.2   0:00.01 S          `- sh                                                                                                                                                                                                        
  372 root      20   0    2.1m   1.2m   0.0  0.2   0:00.32 S      `- dropbear                                                                                                                                                                                                      
  373 root      20   0    1.9m   1.5m   0.0  0.2   0:00.00 S          `- sh                                                                                                                                                                                                        
  375 root      20   0    2.5m   1.5m   0.7  0.2   0:00.98 R              `- top                                                                                                                                                                                                   
  277 root      20   0  240.4m  14.5m   0.0  2.0   0:00.34 S  `- webbridge                                                                                                                                                                                                         
  301 root      20   0  182.7m  19.2m   3.9  2.6   0:02.15 S      `- rpcprocess                                                                                                                                                                                                    
  312 root      20   0  198.9m  26.7m  80.9  3.6   0:39.09 R          `- WPENetworkProce                                                                                                                                                                                           
  316 root      20   0  410.8m 111.5m  44.1 15.1   0:34.95 R          `- WPEWebProcess                                                                                                                                                                                             
  315 root      20   0   56.5m   3.2m   0.0  0.4   0:00.05 S      `- rpcprocess ```

[vertexodessa]

FWIW I tried to collect CPU profile with WPE Webkit running on PC, seeing mostly SSL-related calls occupying CPU in NetworkProcess

[zdobersek]

Posted analysis here: https://bugs.webkit.org/show_bug.cgi?id=167420

This has to be further reported to GnuTLS.

[blino]

Also note that you can use glib-openssl instead of GnuTLS, to avoid deps that may not be suitable for a product.

[zdobersek]

Reported upstream, and is actively being worked on: https://gitlab.com/gnutls/gnutls/issues/165

[zak211]

Verified the issue using gnutls 3.3.22 + the backporting patch https://gitlab.com/gnutls/gnutls/commit/1a5e67d6bd80afe75ec78b3e9a002ae1130ca089 Issue is not reproducible anymore close #169