Add blur, shadow, and corner radius to layer-shell surfaces

[ErikReider]

Adds border radius and blur to layer-shell surfaces (not background and bottom layers).

Requires #157

Todo:

• [x] Command to add layer namespace with effects

• [x] Handle blur

• [x] Handle shadow

• [x] Handle corner radius

• [x] Add layer-effect checking in get_workspace_effect_info

• [x] Change config command syntax to match for_window

• [x] Add method of viewing all layer namespaces in sway-ipc

• [x] Make the command syntax similar to for_window

• [x] Code cleanup (change the config syntax?)

• [x] Disable blur xray for bottom layer surfaces

Example command: layer_effects "waybar" blur enable; shadows enable; corner_radius 6

[ErikReider]

~I skipped adding shadows due to them only being rendered for the base surface. Would be easier if we render shadows in render_surface_iterator. Shadows could be moved into there in this commit if desired :)~

Other than that and the discussion todos, this should be ready

Edit: I added support for shadows and it works really well so let me know if it's something better suited for a separate PR

[ErikReider]

Seems like moving the shadows into render_surface_iterator fixes #141 :)

[ErikReider]

I updated the description with more discussion topics :)

[WillPower3309]

Moving to render_surface_iterator breaks the smart_corner_radius shadow fixes: https://github.com/WillPower3309/swayfx/commit/74e7012b21d48200cf67608f965befe5a308a8bb

I'll extract the render_surface_iterator changes to a different PR to help focus on this piece by piece and ensure nothing breaks along the way

Edit: #150

[WillPower3309]

shadows are very view dependent, maybe we keep them in render_view and then have a oneliner for layer surfaces here since no other processing (other than corner_radius scale) needs to be done:

render_box_shadow(output, output_damage, &shadow_box, config->shadow_color, config->shadow_blur_sigma, corner_radius);

We've shown in #150 (sadly) that moving the shadows to render_surface_iterator may not have been what fixed #141

@ErikReider if this is something that you agree with, I can make this change

[ErikReider]

Layers on the bottom layer shouldn't be able to blur (at least with xray on) due to how the blur optimization currently works. The result is that it blurs itself. (The bottom layer is drawn before the blur optimization and the tiled windows are drawn)

[ErikReider]

Moving to render_surface_iterator breaks the smart_corner_radius shadow fixes: 74e7012

Why wouldn't we be able to move that logic into render_surface_iterator? All of the needed pointers are available for tiled windows (when view isn't null). Or did I misunderstand what you wrote? 😅

[WillPower3309]

Moving to render_surface_iterator breaks the smart_corner_radius shadow fixes: 74e7012

Why wouldn't we be able to move that logic into render_surface_iterator? All of the needed pointers are available for tiled windows (when view isn't null). Or did I misunderstand what you wrote? 😅

Was just a bug caused by the below line:

corner_radius = (con->corner_radius + state->border_thickness) * wlr_output->scale;

In render_surface iterator. Changed it to be

corner_radius += state->border_thickness

In #150 and then applying the scale after. It was taking the containers corner radius instead of the decoration data's corner radius (which was the one that was updated for the smart gaps)

[WillPower3309]

Will need a hefty rebase with #155 merged

[ErikReider]

Will need a hefty rebase with #155 merged

I'm on it! o7

[WillPower3309]

Rebased + made some changes so we aren't rendering shadows in the surface iterator, and some minor refactors

[WillPower3309]

We do lose xray for layer shell popups here, but if they're really needed we can look into returning that functionality