Extreme memory leak when screenshare but memory not used by any process

[Tofixrs]

Regression?

Yes

System Info and Version

System/Version info

```sh Hyprland, built from branch at commit cc98594c3aed0b542e03818371a4636f549f80e1 (). Date: 2024-07-05 Tag: , commits: cc98594c3aed0b542e03818371a4636f549f80e1 flags: (if any) System Information: System name: Linux Node name: lapfix Release: 6.9.7 Version: #1-NixOS SMP PREEMPT_DYNAMIC Thu Jun 27 11:52:32 UTC 2024 GPU information: 01:00.0 VGA compatible controller [0300]: NVIDIA Corporation GA107M [GeForce RTX 3050 Mobile] [10de:25a2] (rev a1) (prog-if 00 [VGA controller]) 06:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Cezanne [Radeon Vega Series / Radeon Vega Mobile Series] [1002:1638] (rev c6) (prog-if 00 [VGA controller]) NVRM version: NVIDIA UNIX Open Kernel Module for x86_64 555.58.02 Release Build (nixbld@) Wed Jul 3 01:26:00 UTC 2024 os-release: ANSI_COLOR="1;34" BUG_REPORT_URL="https://github.com/NixOS/nixpkgs/issues" BUILD_ID="24.11.20240703.9f4128e" DOCUMENTATION_URL="https://nixos.org/learn.html" HOME_URL="https://nixos.org/" ID=nixos IMAGE_ID="" IMAGE_VERSION="" LOGO="nix-snowflake" NAME=NixOS PRETTY_NAME="NixOS 24.11 (Vicuna)" SUPPORT_URL="https://nixos.org/community.html" VERSION="24.11 (Vicuna)" VERSION_CODENAME=vicuna VERSION_ID="24.11" plugins: ======Config-Start====== Config File: /home/tofix/.config/hypr/hyprland.conf: Read Succeeded exec-once = /nix/store/y29cbwfiwb9nyi4db4apk9dgylzdjm1v-dbus-1.14.10/bin/dbus-update-activation-environment --systemd DISPLAY HYPRLAND_INSTANCE_SIGNATURE WAYLAND_DISPLAY XDG_CURRENT_DESKTOP && systemctl --user stop hyprland-session.target && systemctl --user start hyprland-session.target $mainMod=SUPER animations { bezier=md3_standard, 0.2, 0, 0, 1 bezier=md3_decel, 0.05, 0.7, 0.1, 1 bezier=md3_accel, 0.3, 0, 0.8, 0.15 bezier=overshot, 0.05, 0.9, 0.1, 1.1 bezier=crazyshot, 0.1, 1.5, 0.76, 0.92 bezier=hyprnostretch, 0.05, 0.9, 0.1, 1.0 bezier=fluent_decel, 0.1, 1, 0, 1 bezier=easeInOutCirc, 0.85, 0, 0.15, 1 bezier=easeOutCirc, 0, 0.55, 0.45, 1 animation=windows, 1, 3, md3_decel, popin 60% animation=border, 1, 10, default animation=fade, 1, 2, default animation=workspaces, 1, 3.5, md3_decel, slide animation=specialWorkspace, 1, 3, md3_decel, slidefadevert 15% enabled=true } binds { bind=$mainMod, RETURN, exec, $TERMINAL bind=$mainMod, M, exit bind=$mainMod, C, killactive bind=$mainMod, R, exec, anyrun bind=$mainMod, F, togglefloating bind=$mainMod CONTROL, F, fullscreen, 0 bind=$mainMod, left, movefocus, l bind=$mainMod, right, movefocus, r bind=$mainMod, up, movefocus, u bind=$mainMod, down, movefocus, d bind=$mainMod SHIFT, left, movewindow, l bind=$mainMod SHIFT, right, movewindow, r bind=$mainMod SHIFT, up, movewindow, u bind=$mainMod SHIFT, down, movewindow, d bind=$mainMod, mouse_down, workspace, e+1 bind=$mainMod, mouse_up, workspace, e-1 bind=$mainMod, Print, exec, /nix/store/6h4vwrkbdbbw9ml0y0gy16cawcfkadpa-grimblast-0.1/bin/grimblast --freeze save area - | swappy -f - -o ~/Pictures/screenshots/$(date +'%s_grim.png') bind=, Print, exec, /nix/store/6h4vwrkbdbbw9ml0y0gy16cawcfkadpa-grimblast-0.1/bin/grimblast --freeze save screen - | swappy -f - -o ~/Pictures/screenshots/$(date +'%s_grim.png') bind=$mainMod, TAB, exec, ags --toggle-window dashboard bind=$mainMod, V, exec, ags --toggle-window clipboard bind=$mainMod, X, exec, ags --toggle-window powermenu bind=, xf86audioplay, exec, playerctl play-pause bind=, xf86audionext, exec, playerctl next bind=, xf86audioprev, exec, playerctl previous bind=, xf86audiostop, exec, playerctl stop bind=, xf86audiomute, exec, wpctl set-mute @DEFAULT_SINK@ toggle bind=, xf86monbrightnessup, exec, brightnessctl set 50- bind=, xf86monbrightnessdown, exec, brightnessctl set 50+ bind=, code:179, exec, spotify bind=CTRL, xf86audionext, exec, playerctl position 5 bind=CTRL, xf86audioprev, exec, playerctl position 5 - bind=$mainMod, l, exec, /nix/store/qnhcl0rkipbjkfg78xnq5ki63xhgsj9i-hyprlock-0.3.0-unstable-2024-04-24/bin/hyprlock bind=$mainMod CTRL, C, exec, hyprpicker -a -r -f hex bind=$mainMod, 1, workspace, 1 bind=$mainMod, 2, workspace, 2 bind=$mainMod, 3, workspace, 3 bind=$mainMod, 4, workspace, 4 bind=$mainMod, 5, workspace, 5 bind=$mainMod, 6, workspace, 6 bind=$mainMod, 7, workspace, 7 bind=$mainMod, 8, workspace, 8 bind=$mainMod, 9, workspace, 9 bind=$mainMod, 0, workspace, 10 bind=$mainMod SHIFT, 1, movetoworkspace, 1 bind=$mainMod SHIFT, 2, movetoworkspace, 2 bind=$mainMod SHIFT, 3, movetoworkspace, 3 bind=$mainMod SHIFT, 4, movetoworkspace, 4 bind=$mainMod SHIFT, 5, movetoworkspace, 5 bind=$mainMod SHIFT, 6, movetoworkspace, 6 bind=$mainMod SHIFT, 7, movetoworkspace, 7 bind=$mainMod SHIFT, 8, movetoworkspace, 8 bind=$mainMod SHIFT, 9, movetoworkspace, 9 bind=$mainMod SHIFT, 0, movetoworkspace, 10 bind=$mainMod CONTROL, 1, movetoworkspacesilent, 1 bind=$mainMod CONTROL, 2, movetoworkspacesilent, 2 bind=$mainMod CONTROL, 3, movetoworkspacesilent, 3 bind=$mainMod CONTROL, 4, movetoworkspacesilent, 4 bind=$mainMod CONTROL, 5, movetoworkspacesilent, 5 bind=$mainMod CONTROL, 6, movetoworkspacesilent, 6 bind=$mainMod CONTROL, 7, movetoworkspacesilent, 7 bind=$mainMod CONTROL, 8, movetoworkspacesilent, 8 bind=$mainMod CONTROL, 9, movetoworkspacesilent, 9 bind=$mainMod CONTROL, 0, movetoworkspacesilent, 10 binde=$mainMod CONTROL, up, resizeactive, 0 -20 binde=$mainMod CONTROL, down, resizeactive, 0 20 binde=$mainMod CONTROL, left, resizeactive, -20 0 binde=$mainMod CONTROL, right, resizeactive, 20 0 binde=, xf86audiolowervolume, exec, wpctl set-volume @DEFAULT_SINK@ 5%- binde=, xf86audioraisevolume, exec, wpctl set-volume @DEFAULT_SINK@ 5%+ bindm=$mainMod, mouse:272, movewindow bindm=$mainMod, mouse:273, resizewindow } debug { disable_logs=false } decoration { col.shadow=rgb(11111b) inactive_opacity=0.900000 rounding=5 } dwindle { preserve_split=true pseudotile=true } general { border_size=2 col.active_border=rgb(cba6f7) rgb(f38ba8) 45deg col.inactive_border=rgb(313244) gaps_in=2.500000 gaps_out=7 } gestures { workspace_swipe=true workspace_swipe_create_new=true workspace_swipe_forever=true } input { touchpad { natural_scroll=true } kb_layout=pl numlock_by_default=true } master { new_status=master } misc { disable_hyprland_logo=true vrr=1 } exec-once=ags exec-once=sleep 1 && swww init && swww img /home/tofix/wallpaper.jpg exec-once=keepassxc monitor=eDP-1,1920x1080@144,0x0,1,vrr,1 windowrulev2=float, class:^(org.kde.polkit-kde-authentication-agent-1)$ windowrulev2=float, class:^(Steam)$ title:^(Friends List)$ windowrulev2=size 500 500, class:^(Steam)$ title:^(Friends List)$ windowrulev2=float, class:^(steam)$ title:^(Friends List)$ windowrulev2=size 500 500, class:^(steam)$ title:^(Friends List)$ windowrulev2=size 930 495, class:^(firefox)$ title:^(Picture-in-Picture)$ windowrulev2=float, class:^(firefox)$ title:^(Picture-in-Picture)$ windowrulev2=suppressevent, fullscreen, class:^(firefox)$ title:^(Picture-in-Picture)$ windowrulev2=suppressevent, fullscreen, class:^(libreoffice-startcenter)$ windowrulev2=float, class:^(firefox)$, title:^(Firefox — Sharing Indicator)$ windowrulev2=suppressevent, fullscreen, class:^(firefox)$, title:^(Firefox — Sharing Indicator)$ windowrulev2=suppressevent, maximazie, class:^(firefox)$, title:^(Firefox — Sharing Indicator)$ windowrulev2=move 0 0, class:^(firefox)$, title:^(Firefox — Sharing Indicator)$ windowrulev2=suppressevent, fullscreen, class:^(firefox)$, title:^(Firefox — Sharing Indicator)$ windowrulev2=noinitialfocus, class:^(steam)$, title:^(notificationtoasts) windowrulev2=float, class:^(com-atlauncher-App)$ title:^(ATLauncher Console)$ windowrulev2=float, class:^(org.keepassxc.KeePassXC)$ title: (Unlock) windowrulev2=tile,class:^dev.warp.Warp.*$ windowrulev2=float, class:^(org.keepassxc.KeePassXC)$ title: (Passkey credentials) ======Config-End======== ```

Description

Anytime i screen share my memory usage starts ramping up but it seems like no process is using said memory. It continues to fill up the memory until the system stops responding unless i stop screen capture The only way i can free up the memory is to reboot the system

How to reproduce

Just screen share anything

Crash reports, logs, images, videos

[Tofixrs]

Gonna be bisecting in a bit

[Tofixrs]

Also it seems like the colors are messed up when i capture a window

Edit: red and blue are swapped

[Tofixrs]

After a bit more examination this only seems to happen when i have an external monitor plugged in into my laptop

[ikalco]

same as #6772 ?

[vaxerski]

ye