search

canonical / ubuntu-frame

The foundation for many embedded graphical display implementations
GNU General Public License v3.0
156 stars 21 forks source link

Hardware acceleration on Odroid M1 #159

Closed Elevate-IT closed 9 months ago

Elevate-IT commented 9 months ago

Hello,

I'm currently struggling to get hardware acceleration working on my Odroid M1

Hardware: Odroid M1 Rockchip RK3568 Processor Mali-G52 MP2 GPU

Software: Ubuntu server 22.04 LTS ubuntu-frame ubuntu-frame-osk ubuntu-frame-vnc wpe-webkit-mir-kiosk

The system works but wpe-webkit-mir-kiosk restarts every few minutes with error: DRM_IOCTL_PANFROST_MMAP_BO failed: No such file or directory

When I run "WAYLAND_DISPLAY=wayland-99 ubuntu-frame" I get this output:

+ mkdir -p /home/odroid/snap/ubuntu-frame/7027/.config/
+ grep -vE (^vt=|^console-provider=vt) /var/snap/ubuntu-frame/7027/frame.config
+ rm -f /home/odroid/snap/ubuntu-frame/7027/.config/frame.display
+ cp /var/snap/ubuntu-frame/7027/frame.display /home/odroid/snap/ubuntu-frame/7027/.config/
+ exec /snap/ubuntu-frame/7027/bin/run-frame /snap/ubuntu-frame/7027/bin/graphics-core22-wrapper /snap/ubuntu-frame/7027/usr/local/bin/frame
+ [ /snap/ubuntu-frame/7027/bin/graphics-core22-wrapper = --help ]
+ dirname /run/user/1000/snap.ubuntu-frame
+ XDG_RUNTIME_DIR=/run/user/1000
+ export XDG_RUNTIME_DIR
+ mkdir -p /run/user/1000 -m 700
+ [ -n wayland-99 ]
+ [ -e /run/user/1000/wayland-99 ]
+ exec /snap/ubuntu-frame/7027/bin/graphics-core22-wrapper /snap/ubuntu-frame/7027/usr/local/bin/frame
[2023-11-15 14:57:43.682917] < - debug - > miral: Loaded display configuration file: /home/odroid/snap/ubuntu-frame/7027/.config/frame.display
[2023-11-15 14:57:43.734014] <information> mirserver: Starting
[2023-11-15 14:57:43.819939] < - debug - > mirserver: Not using logind for session management: Seat has no active session
[2023-11-15 14:57:43.820524] < - debug - > mirserver: Not using Linux VT subsystem for session management: Failed to find the current VT
[2023-11-15 14:57:43.820672] < - debug - > mirserver: No session management supported
[2023-11-15 14:57:43.820795] <information> VT switch key handler: No VT switching support available: MinimalConsoleServices does not support VT switching
[2023-11-15 14:57:43.821600] <information> mircommon: Loading modules from: /snap/ubuntu-frame/7027/usr/lib/aarch64-linux-gnu/mir/server-platform
[2023-11-15 14:57:43.822000] <information> mircommon: Loading module: /snap/ubuntu-frame/7027/usr/lib/aarch64-linux-gnu/mir/server-platform/graphics-eglstream-kms.so.20
[2023-11-15 14:57:43.822191] <information> mircommon: Loading module: /snap/ubuntu-frame/7027/usr/lib/aarch64-linux-gnu/mir/server-platform/graphics-gbm-kms.so.20
[2023-11-15 14:57:43.822321] <information> mircommon: Loading module: /snap/ubuntu-frame/7027/usr/lib/aarch64-linux-gnu/mir/server-platform/renderer-egl-generic.so.20
[2023-11-15 14:57:43.822422] <information> mircommon: Loading module: /snap/ubuntu-frame/7027/usr/lib/aarch64-linux-gnu/mir/server-platform/server-x11.so.20
[2023-11-15 14:57:43.822515] <information> mircommon: Loading module: /snap/ubuntu-frame/7027/usr/lib/aarch64-linux-gnu/mir/server-platform/input-evdev.so.8
[2023-11-15 14:57:43.827034] <information> mirserver: Found display driver: mir:eglstream-kms (version 2.15.0)
[2023-11-15 14:57:43.833326] < - debug - > eglstream: Found EGLDeviceEXT with device extensions: EGL_EXT_device_drm EGL_EXT_device_drm_render_node
[2023-11-15 14:57:43.834063] <information> eglstream: Failed to query DRM node for EGLDevice: Failed to acquire DRM master: Operation not permitted
[2023-11-15 14:57:43.834164] < - debug - > eglstream: Found EGLDeviceEXT with device extensions: EGL_MESA_device_software EGL_EXT_device_drm_render_node
[2023-11-15 14:57:43.834332] <information> eglstream: Failed to query DRM node for EGLDevice: Failed to determine DRM device node path from EGLDevice: EGL_BAD_PARAMETER (0x300c)
[2023-11-15 14:57:43.834376] < - debug - > eglstream: EGLDeviceEXTs found, but none are suitable for Mir
[2023-11-15 14:57:43.834418] <information> mirserver: (Unsupported by system environment)
[2023-11-15 14:57:43.834464] <information> mirserver: Found display driver: mir:gbm-kms (version 2.15.0)
[2023-11-15 14:57:43.836694] < - debug - > gbm-kms: Quirks: checking device with devnode: /dev/dri/card1, driver rockchip-drm
[2023-11-15 14:57:43.837209] <information> gbm-kms: Failed to probe DRM device: ./src/server/console/minimal_console_services.cpp(134): Throw in function virtual std::future<std::unique_ptr<mir::Device> > mir::MinimalConsoleServices::acquire_device(int, int, std::unique_ptr<mir::Device::Observer>)
Dynamic exception type: boost::wrapexcept<std::system_error>
std::exception::what: Failed to acquire DRM master: Operation not permitted

[2023-11-15 14:57:43.837703] < - debug - > gbm-kms: Quirks: checking device with devnode: (null), driver
[2023-11-15 14:57:43.838146] < - debug - > gbm-kms: Quirks: checking device with devnode: /dev/dri/card0, driver panfrost
[2023-11-15 14:57:43.838569] <information> gbm-kms: Failed to probe DRM device: ./src/server/console/minimal_console_services.cpp(134): Throw in function virtual std::future<std::unique_ptr<mir::Device> > mir::MinimalConsoleServices::acquire_device(int, int, std::unique_ptr<mir::Device::Observer>)
Dynamic exception type: boost::wrapexcept<std::system_error>
std::exception::what: Failed to acquire DRM master: Operation not permitted

[2023-11-15 14:57:43.838776] <information> mirserver: (Unsupported by system environment)
[2023-11-15 14:57:43.838992] <information> mirserver: Found display driver: mir:x11 (version 2.15.0)
[2023-11-15 14:57:43.839103] <information> mirserver: (Unsupported by system environment)
ERROR: ./src/server/graphics/default_configuration.cpp(233): Throw in function virtual const std::vector<std::shared_ptr<mir::graphics::DisplayPlatform> >& mir::DefaultServerConfiguration::the_display_platforms()
Dynamic exception type: boost::wrapexcept<std::runtime_error>
std::exception::what: Exception while creating graphics platform
ERROR: ./src/server/graphics/platform_probe.cpp(190): Throw in function std::vector<std::pair<mir::graphics::SupportedDevice, std::shared_ptr<mir::SharedLibrary> > > {anonymous}::modules_for_device({anonymous}::ModuleType, const std::vector<std::shared_ptr<mir::SharedLibrary> >&, const mir::options::ProgramOption&, const std::shared_ptr<mir::ConsoleServices>&)
Dynamic exception type: boost::wrapexcept<std::runtime_error>
std::exception::what: Failed to find any platforms for current system

Is there a way to have hardware acceleration working or is this GPU simply not supported?

Thanks in advance!

AlanGriffiths commented 9 months ago

@Elevate-IT the log you provide doesn't help.

It shows that Frame is failing to start, but the reason for that is that it has neither an active logind session nor root and therefore cannot take control of the graphics card.

From your hardware, I believe that the Mesa Lima driver will work with Frame.

By "Hardware acceleration" I assume you mean video decoding? That isn't a requirement for Frame but may be used by wpe-webkit-mir-kiosk.

The relevant logs can be retrieved by:

snap logs -fn 100 wpe-webkit-mir-kiosk

For support on wpe-webkit-mir-kiosk

$ snap info wpe-webkit-mir-kiosk | grep contact
contact:   kontakt@glancr.de
Saviq commented 9 months ago

@Elevate-IT also, https://snapcraft.io/graphics-test-tools/ are a set of utilities that could help show what works, what doesn't.

Elevate-IT commented 9 months ago

First of al thanks for the quick response!

I started with a fresh ubuntu 22.04 LTS installation. Installed all the necessary snap packages and everything seems to work now. (it doesn't crash anymore)

I think the difference is the kernel version. currently I'm on '5.18.0-odroid-arm64' Yesterday I installed '6.1.0-odroid-arm64'

'snap logs wpe-webkit-mir-kiosk' I can see some ALSA errors. Currently we don't need audio. Is there an easy fix of can I turn this off? (to clean up the logs)

2023-11-16T11:40:38+01:00 wpe-webkit-mir-kiosk.daemon[4397]: Cannot connect to server socket err = No such file or directory
2023-11-16T11:40:38+01:00 wpe-webkit-mir-kiosk.daemon[4397]: Cannot connect to server request channel
2023-11-16T11:40:38+01:00 wpe-webkit-mir-kiosk.daemon[4397]: jack server is not running or cannot be started
2023-11-16T11:40:38+01:00 wpe-webkit-mir-kiosk.daemon[4397]: JackShmReadWritePtr::~JackShmReadWritePtr - Init not done for -1, skipping unlock
2023-11-16T11:40:38+01:00 wpe-webkit-mir-kiosk.daemon[4397]: JackShmReadWritePtr::~JackShmReadWritePtr - Init not done for -1, skipping unlock
2023-11-16T11:40:38+01:00 wpe-webkit-mir-kiosk.daemon[4397]: ALSA lib conf.c:4120:(snd_config_update_r) Cannot access file /usr/share/alsa/alsa.conf
2023-11-16T11:40:38+01:00 wpe-webkit-mir-kiosk.daemon[4397]: ALSA lib pcm.c:2642:(snd_pcm_open_noupdate) Unknown PCM default
2023-11-16T11:40:38+01:00 wpe-webkit-mir-kiosk.daemon[4397]: AL lib: (EE) ALCplaybackAlsa_open: Could not open playback device 'default': No such file or directory

'snap logs ubuntu-frame'

2023-11-16T11:40:50+01:00 ubuntu-frame.daemon[4126]: [2023-11-16 11:40:50.997137] < -warning- > mirserver: Ignoring layer shell protocol violation: wl_surface destroyed before associated zwlr_layer_surface_v1@28
2023-11-16T11:40:51+01:00 ubuntu-frame.daemon[4126]: [2023-11-16 11:40:51.285049] <information> frame: aa_getpeercon() failed for process 4364: Invalid argument
2023-11-16T11:40:51+01:00 ubuntu-frame.daemon[4126]: [2023-11-16 11:40:51.285194] <information> frame: Fall back (without AppArmor): Identify client via /proc/%d/cmdline
2023-11-16T11:40:53+01:00 ubuntu-frame.daemon[4126]: [2023-11-16 11:40:53.156386] <information> frame: aa_getpeercon() failed for process 4364: Invalid argument
2023-11-16T11:40:53+01:00 ubuntu-frame.daemon[4126]: [2023-11-16 11:40:53.156539] <information> frame: Fall back (without AppArmor): Identify client via /proc/%d/cmdline

How can i fix these errors?

'aa-status' gives me

apparmor module is loaded.
apparmor filesystem is not mounted.
Elevate-IT commented 9 months ago

After some time the error returns. Here are some logs.

snap logs wpe-webkit-mir-kiosk

2023-11-28T10:49:13+01:00 -[19537]: <https://google.be> Loaded successfully.
2023-11-28T10:49:21+01:00 wpe-webkit-mir-kiosk.daemon[19537]: DRM_IOCTL_PANFROST_MMAP_BO failed: No such file or directory
2023-11-28T10:49:21+01:00 wpe-webkit-mir-kiosk.daemon[19537]: mmap failed: result=(nil) size=0x7f8000 fd=13 offset=0x0 Invalid argument
2023-11-28T10:49:21+01:00 wpe-webkit-mir-kiosk.daemon[19537]: **
2023-11-28T10:49:21+01:00 wpe-webkit-mir-kiosk.daemon[19537]: Cog-Wayland:ERROR:../src/platform/wayland/cog-platform-wl.c:1681:on_export_wl_egl_image: assertion failed: (wpe_view_data.buffer)
2023-11-28T10:49:21+01:00 wpe-webkit-mir-kiosk.daemon[19537]: Bail out! Cog-Wayland:ERROR:../src/platform/wayland/cog-platform-wl.c:1681:on_export_wl_egl_image: assertion failed: (wpe_view_data.buffer)
2023-11-28T10:49:24+01:00 wpe-webkit-mir-kiosk.daemon[19440]: /snap/wpe-webkit-mir-kiosk/101/bin/launch-wpe: line 48: 19537 Aborted                 (core dumped) "$SNAP"/usr/bin/cog "${cog_params[@]}" "$url"
2023-11-28T10:49:24+01:00 systemd[1]: snap.wpe-webkit-mir-kiosk.daemon.service: Main process exited, code=exited, status=134/n/a
2023-11-28T10:49:24+01:00 systemd[1]: snap.wpe-webkit-mir-kiosk.daemon.service: Failed with result 'exit-code'.
2023-11-28T10:49:24+01:00 systemd[1]: snap.wpe-webkit-mir-kiosk.daemon.service: Consumed 1min 11.677s CPU time.
2023-11-28T10:49:27+01:00 systemd[1]: snap.wpe-webkit-mir-kiosk.daemon.service: Scheduled restart job, restart counter is at 5.
2023-11-28T10:49:27+01:00 systemd[1]: Stopped Service for snap application wpe-webkit-mir-kiosk.daemon.
2023-11-28T10:49:27+01:00 systemd[1]: snap.wpe-webkit-mir-kiosk.daemon.service: Consumed 1min 11.677s CPU time.
2023-11-28T10:49:27+01:00 systemd[1]: Started Service for snap application wpe-webkit-mir-kiosk.daemon.
2023-11-28T10:49:28+01:00 wpe-webkit-mir-kiosk.daemon[19791]: Could not determine the accessibility bus address
2023-11-28T10:49:28+01:00 -[19791]: Your application does not implement g_application_activate() and has no handlers connected to the 'activate' signal.  It should do one of these.
2023-11-28T10:49:28+01:00 -[19791]: Acquired D-Bus well-known name com.igalia.Cog
2023-11-28T10:49:28+01:00 -[19791]: <https://google.be> Load started.

Driver wise everything looks ok I think?

admin@Kiosk1:~$  lsmod | grep panfrost
panfrost               69632  5
gpu_sched              40960  1 panfrost
drm_shmem_helper       28672  1 panfrost
admin@Kiosk1:~$ dmesg | grep -i panfrost
[    3.156379] panfrost fde60000.gpu: clock rate = 594000000
[    3.156406] panfrost fde60000.gpu: bus_clock rate = 500000000
[    3.166692] panfrost fde60000.gpu: mali-g52 id 0x7402 major 0x1 minor 0x0 status 0x0
[    3.166707] panfrost fde60000.gpu: features: 00000000,00000cf7, issues: 00000000,00000400
[    3.166713] panfrost fde60000.gpu: Features: L2:0x07110206 Shader:0x00000002 Tiler:0x00000209 Mem:0x1 MMU:0x00002823 AS:0xff JS:0x7
[    3.166719] panfrost fde60000.gpu: shader_present=0x1 l2_present=0x1
[    3.173993] [drm] Initialized panfrost 1.2.0 20180908 for fde60000.gpu on minor 1

There is not a lot of info to find about this error.

admin@Kiosk1:~$ sudo journalctl -xe | grep Cog-Wayland
Nov 28 10:32:31 C214-Kiosk1 wpe-webkit-mir-kiosk.daemon[18502]: Cog-Wayland:ERROR:../src/platform/wayland/cog-platform-wl.c:1681:on_export_wl_egl_image: assertion failed: (wpe_view_data.buffer)
Nov 28 10:32:31 C214-Kiosk1 wpe-webkit-mir-kiosk.daemon[18502]: Bail out! Cog-Wayland:ERROR:../src/platform/wayland/cog-platform-wl.c:1681:on_export_wl_egl_image: assertion failed: (wpe_view_data.buffer)
Nov 28 10:37:19 C214-Kiosk1 wpe-webkit-mir-kiosk.daemon[18837]: Cog-Wayland:ERROR:../src/platform/wayland/cog-platform-wl.c:1681:on_export_wl_egl_image: assertion failed: (wpe_view_data.buffer)
Nov 28 10:37:19 C214-Kiosk1 wpe-webkit-mir-kiosk.daemon[18837]: Bail out! Cog-Wayland:ERROR:../src/platform/wayland/cog-platform-wl.c:1681:on_export_wl_egl_image: assertion failed: (wpe_view_data.buffer)
Nov 28 10:40:34 C214-Kiosk1 wpe-webkit-mir-kiosk.daemon[19120]: Cog-Wayland:ERROR:../src/platform/wayland/cog-platform-wl.c:1681:on_export_wl_egl_image: assertion failed: (wpe_view_data.buffer)
Nov 28 10:40:34 C214-Kiosk1 wpe-webkit-mir-kiosk.daemon[19120]: Bail out! Cog-Wayland:ERROR:../src/platform/wayland/cog-platform-wl.c:1681:on_export_wl_egl_image: assertion failed: (wpe_view_data.buffer)
Nov 28 10:44:25 C214-Kiosk1 wpe-webkit-mir-kiosk.daemon[19338]: Cog-Wayland:ERROR:../src/platform/wayland/cog-platform-wl.c:1681:on_export_wl_egl_image: assertion failed: (wpe_view_data.buffer)
Nov 28 10:44:25 C214-Kiosk1 wpe-webkit-mir-kiosk.daemon[19338]: Bail out! Cog-Wayland:ERROR:../src/platform/wayland/cog-platform-wl.c:1681:on_export_wl_egl_image: assertion failed: (wpe_view_data.buffer)
Nov 28 10:49:21 C214-Kiosk1 wpe-webkit-mir-kiosk.daemon[19537]: Cog-Wayland:ERROR:../src/platform/wayland/cog-platform-wl.c:1681:on_export_wl_egl_image: assertion failed: (wpe_view_data.buffer)
Nov 28 10:49:21 C214-Kiosk1 wpe-webkit-mir-kiosk.daemon[19537]: Bail out! Cog-Wayland:ERROR:../src/platform/wayland/cog-platform-wl.c:1681:on_export_wl_egl_image: assertion failed: (wpe_view_data.buffer)
admin@Kiosk1:~$ snap list
Name                  Version                 Rev    Tracking       Publisher   Notes
core20                20230801                2019   latest/stable  canonical✓  base
core22                20230801                867    latest/stable  canonical✓  base
mesa-core20           21.2.6                  154    latest/stable  canonical✓  -
mesa-core22           23.0.4                  234    latest/stable  canonical✓  -
snapd                 2.60.4                  20298  latest/stable  canonical✓  snapd
ubuntu-frame          118-mir2.15.0           7027   22/stable      canonical✓  -
ubuntu-frame-osk      49-squeekboard-v1.17.1  374    22/stable      canonical✓  -
ubuntu-frame-vnc      45-wayvncv0.6.2         228    22/stable      canonical✓  -
wpe-webkit-mir-kiosk  2.38.5                  101    latest/stable  glancr      -
admin@Kiosk1:~$ snap connections wpe-webkit-mir-kiosk
Interface                 Plug                                   Slot                              Notes
audio-playback            wpe-webkit-mir-kiosk:audio-playback    :audio-playback                   -
avahi-observe             wpe-webkit-mir-kiosk:avahi-observe     -                                 -
content[graphics-core20]  wpe-webkit-mir-kiosk:graphics-core20   mesa-core20:graphics-core20       -
dbus                      -                                      wpe-webkit-mir-kiosk:dbus-cogctl  -
hardware-observe          wpe-webkit-mir-kiosk:hardware-observe  -                                 -
hostname-control          wpe-webkit-mir-kiosk:hostname-control  -                                 -
network                   wpe-webkit-mir-kiosk:network           :network                          -
network-bind              wpe-webkit-mir-kiosk:network-bind      :network-bind                     -
network-manager           wpe-webkit-mir-kiosk:network-manager   -                                 -
opengl                    wpe-webkit-mir-kiosk:opengl            :opengl                           -
process-control           wpe-webkit-mir-kiosk:process-control   -                                 -
system-observe            wpe-webkit-mir-kiosk:system-observe    -                                 -
upower-observe            wpe-webkit-mir-kiosk:upower-observe    :upower-observe                   -
wayland                   wpe-webkit-mir-kiosk:wayland           :wayland                          manual

I tried reinstalling all snaps without success.

Could somebody point me in the right direction? The system worked perfectly for more than a week.

Thanks in advance!

Saviq commented 9 months ago

Hi @Elevate-IT, the lack of apparmor (lack of support in the kernel) is unfortunate, but not critical. That's why you get the aa_getpeercon warning. Sound - no, no easy way to turn it off.

On the graphics side, can you successfully use e.g. eglinfo from outside a snap?

Could it be you're running out of memory? This seems to be the first indication of things going wrong - search for this bit:

DRM_IOCTL_PANFROST_MMAP_BO failed: No such file or directory

Elevate-IT commented 9 months ago

Hi @Saviq

After installing mesa-utils This is the output of eglinfo

admin@Kiosk1:~$ eglinfo
EGL client extensions string:
    EGL_EXT_device_base EGL_EXT_device_enumeration EGL_EXT_device_query
    EGL_EXT_platform_base EGL_KHR_client_get_all_proc_addresses
    EGL_EXT_client_extensions EGL_KHR_debug EGL_EXT_platform_device
    EGL_EXT_platform_wayland EGL_KHR_platform_wayland
    EGL_EXT_platform_x11 EGL_KHR_platform_x11 EGL_EXT_platform_xcb
    EGL_MESA_platform_gbm EGL_KHR_platform_gbm
    EGL_MESA_platform_surfaceless

GBM platform:
MESA-LOADER: failed to open zink: /usr/lib/dri/zink_dri.so: cannot open shared object file: No such file or directory (search paths /usr/lib/aarch64-linux-gnu/dri:\$${ORIGIN}/dri:/usr/lib/dri, suffix _dri)
EGL API version: 1.5
EGL vendor string: Mesa Project
EGL version string: 1.5
EGL client APIs: OpenGL OpenGL_ES 
EGL extensions string:
    EGL_ANDROID_blob_cache EGL_EXT_buffer_age
    EGL_EXT_create_context_robustness EGL_EXT_image_dma_buf_import
    EGL_EXT_image_dma_buf_import_modifiers EGL_KHR_cl_event2
    EGL_KHR_config_attribs EGL_KHR_context_flush_control
    EGL_KHR_create_context EGL_KHR_create_context_no_error
    EGL_KHR_fence_sync EGL_KHR_get_all_proc_addresses
    EGL_KHR_gl_colorspace EGL_KHR_gl_renderbuffer_image
    EGL_KHR_gl_texture_2D_image EGL_KHR_gl_texture_3D_image
    EGL_KHR_gl_texture_cubemap_image EGL_KHR_image EGL_KHR_image_base
    EGL_KHR_image_pixmap EGL_KHR_no_config_context EGL_KHR_reusable_sync
    EGL_KHR_surfaceless_context EGL_EXT_pixel_format_float
    EGL_KHR_wait_sync EGL_MESA_configless_context
    EGL_MESA_image_dma_buf_export EGL_MESA_query_driver
Configurations:
     bf lv colorbuffer dp st  ms    vis   cav bi  renderable  supported
  id sz  l  r  g  b  a th cl ns b    id   eat nd gl es es2 vg surfaces 
---------------------------------------------------------------------
0x01 32  0 10 10 10  2  0  0  0 0 0x30335241--         y  y  y     win
0x02 32  0 10 10 10  2 16  0  0 0 0x30335241--         y  y  y     win
0x03 32  0 10 10 10  2 24  0  0 0 0x30335241--         y  y  y     win
0x04 32  0 10 10 10  2 24  8  0 0 0x30335241--         y  y  y     win
0x05 32  0 10 10 10  2 32  0  0 0 0x30335241--         y  y  y     win
0x06 32  0 10 10 10  2  0  0  4 1 0x30335241--         y  y  y     win
0x07 32  0 10 10 10  2 16  0  4 1 0x30335241--         y  y  y     win
0x08 32  0 10 10 10  2 24  0  4 1 0x30335241--         y  y  y     win
0x09 32  0 10 10 10  2 24  8  4 1 0x30335241--         y  y  y     win
0x0a 32  0 10 10 10  2 32  0  4 1 0x30335241--         y  y  y     win
0x0b 30  0 10 10 10  0  0  0  0 0 0x30335258--         y  y  y     win
0x0c 30  0 10 10 10  0 16  0  0 0 0x30335258--         y  y  y     win
0x0d 30  0 10 10 10  0 24  0  0 0 0x30335258--         y  y  y     win
0x0e 30  0 10 10 10  0 24  8  0 0 0x30335258--         y  y  y     win
0x0f 30  0 10 10 10  0 32  0  0 0 0x30335258--         y  y  y     win
0x10 30  0 10 10 10  0  0  0  4 1 0x30335258--         y  y  y     win
0x11 30  0 10 10 10  0 16  0  4 1 0x30335258--         y  y  y     win
0x12 30  0 10 10 10  0 24  0  4 1 0x30335258--         y  y  y     win
0x13 30  0 10 10 10  0 24  8  4 1 0x30335258--         y  y  y     win
0x14 30  0 10 10 10  0 32  0  4 1 0x30335258--         y  y  y     win
0x15 32  0  8  8  8  8  0  0  0 0 0x34325241--         y  y  y     win
0x16 32  0  8  8  8  8 16  0  0 0 0x34325241--         y  y  y     win
0x17 32  0  8  8  8  8 24  0  0 0 0x34325241--         y  y  y     win
0x18 32  0  8  8  8  8 24  8  0 0 0x34325241--         y  y  y     win
0x19 32  0  8  8  8  8 32  0  0 0 0x34325241--         y  y  y     win
0x1a 32  0  8  8  8  8  0  0  4 1 0x34325241--         y  y  y     win
0x1b 32  0  8  8  8  8 16  0  4 1 0x34325241--         y  y  y     win
0x1c 32  0  8  8  8  8 24  0  4 1 0x34325241--         y  y  y     win
0x1d 32  0  8  8  8  8 24  8  4 1 0x34325241--         y  y  y     win
0x1e 32  0  8  8  8  8 32  0  4 1 0x34325241--         y  y  y     win
0x1f 24  0  8  8  8  0  0  0  0 0 0x34325258--         y  y  y     win
0x20 24  0  8  8  8  0 16  0  0 0 0x34325258--         y  y  y     win
0x21 24  0  8  8  8  0 24  0  0 0 0x34325258--         y  y  y     win
0x22 24  0  8  8  8  0 24  8  0 0 0x34325258--         y  y  y     win
0x23 24  0  8  8  8  0 32  0  0 0 0x34325258--         y  y  y     win
0x24 24  0  8  8  8  0  0  0  4 1 0x34325258--         y  y  y     win
0x25 24  0  8  8  8  0 16  0  4 1 0x34325258--         y  y  y     win
0x26 24  0  8  8  8  0 24  0  4 1 0x34325258--         y  y  y     win
0x27 24  0  8  8  8  0 24  8  4 1 0x34325258--         y  y  y     win
0x28 24  0  8  8  8  0 32  0  4 1 0x34325258--         y  y  y     win
0x29 16  0  5  6  5  0  0  0  0 0 0x36314752--         y  y  y     win
0x2a 16  0  5  6  5  0 16  0  0 0 0x36314752--         y  y  y     win
0x2b 16  0  5  6  5  0 24  0  0 0 0x36314752--         y  y  y     win
0x2c 16  0  5  6  5  0 24  8  0 0 0x36314752--         y  y  y     win
0x2d 16  0  5  6  5  0 32  0  0 0 0x36314752--         y  y  y     win
0x2e 16  0  5  6  5  0  0  0  4 1 0x36314752--         y  y  y     win
0x2f 16  0  5  6  5  0 16  0  4 1 0x36314752--         y  y  y     win
0x30 16  0  5  6  5  0 24  0  4 1 0x36314752--         y  y  y     win
0x31 16  0  5  6  5  0 24  8  4 1 0x36314752--         y  y  y     win
0x32 16  0  5  6  5  0 32  0  4 1 0x36314752--         y  y  y     win
0x33 64  0 16 16 16 16  0  0  0 0 0x48344241--         y  y  y     win
0x34 64  0 16 16 16 16 16  0  0 0 0x48344241--         y  y  y     win
0x35 64  0 16 16 16 16 24  0  0 0 0x48344241--         y  y  y     win
0x36 64  0 16 16 16 16 24  8  0 0 0x48344241--         y  y  y     win
0x37 64  0 16 16 16 16 32  0  0 0 0x48344241--         y  y  y     win
0x38 64  0 16 16 16 16  0  0  4 1 0x48344241--         y  y  y     win
0x39 64  0 16 16 16 16 16  0  4 1 0x48344241--         y  y  y     win
0x3a 64  0 16 16 16 16 24  0  4 1 0x48344241--         y  y  y     win
0x3b 64  0 16 16 16 16 24  8  4 1 0x48344241--         y  y  y     win
0x3c 64  0 16 16 16 16 32  0  4 1 0x48344241--         y  y  y     win
0x3d 48  0 16 16 16  0  0  0  0 0 0x48344258--         y  y  y     win
0x3e 48  0 16 16 16  0 16  0  0 0 0x48344258--         y  y  y     win
0x3f 48  0 16 16 16  0 24  0  0 0 0x48344258--         y  y  y     win
0x40 48  0 16 16 16  0 24  8  0 0 0x48344258--         y  y  y     win
0x41 48  0 16 16 16  0 32  0  0 0 0x48344258--         y  y  y     win
0x42 48  0 16 16 16  0  0  0  4 1 0x48344258--         y  y  y     win
0x43 48  0 16 16 16  0 16  0  4 1 0x48344258--         y  y  y     win
0x44 48  0 16 16 16  0 24  0  4 1 0x48344258--         y  y  y     win
0x45 48  0 16 16 16  0 24  8  4 1 0x48344258--         y  y  y     win
0x46 48  0 16 16 16  0 32  0  4 1 0x48344258--         y  y  y     win

Wayland platform:
eglinfo: eglInitialize failed

X11 platform:
eglinfo: eglInitialize failed

Device platform:
eglinfo: eglInitialize failed

I can confirm that the memory usage is increasing rapidly. It starts at about 700MB and crashes after a few minutes at about 4-5GB. The system has 8GB so it isn't out of memory.

I will look further into "DRM_IOCTL_PANFROST_MMAP_BO failed: No such file or directory"

Elevate-IT commented 9 months ago

Hi @Saviq,

I discovered that this rapid memory increase only happens when I have a VNC connection open to the system. When I close the VNC connection the increase stops and wpe-webkit-mir-kiosk no longer crashes. That also explains why the systems worked perfectly for a week. Because I wasn't looking at it :)

Saviq commented 9 months ago

I discovered that this rapid memory increase only happens when I have a VNC connection open to the system. When I close the VNC connection the increase stops and wpe-webkit-mir-kiosk no longer crashes.

Can you please tell which process is it that seems to leak memory?

Elevate-IT commented 9 months ago

This is the process that increases rapidly. /snap/wpe-webkit-mir-kiosk/101/usr/bin/cog

image

Saviq commented 9 months ago

Huh. That's interesting. VNC shouldn't be affecting WPE's memory usage… We'll need to look into it, hopefully reproduce and track the problem.

In the mean time, you could try the base: core22 build from here:

https://launchpad.net/~mir-team/+snap/wpe-core22-test

I mean to propose it upstream, so WPE maintains the same 20 / 22 / … channels as the rest of the Frame stack.

Elevate-IT commented 9 months ago

After some further testing, I found out that the memory increase only happens when I'm logged in to our web application.

I created a recording so you can see what happens. Video on Dropbox

Will try with base: core22

Saviq commented 9 months ago

After some further testing, I found out that the memory increase only happens when I'm logged in to our web application.

Ah, that makes more sense. This suggests that either your application itself (e.g. leaking objects), or WPE, is leaking memory. Not something that Frame can do anything about.

Please raise this with your application authors and if/when you confirm it doesn't happen on other browsers (such a leak should be visible on any browser), follow up at with https://github.com/Igalia/cog.

I'll close this for Frame as you've confirmed acceleration is working (provided the right kernel pieces).