Add a jitter-free texture filter for pixelart games (sharp bilinear)

[bitbrain]

Describe the project you are working on

I am building an untitled pixelart RPG that has both pixel-perfect scenes but also smooth camera movement.

Describe the problem or limitation you are having in your project

In digital graphics, pixels aren't always displayed one-to-one with screen pixels. For instance, when you zoom in, a sprite might need to cover more screen pixels, or when you zoom out, multiple sprite pixels might be crammed into one screen pixel. To decide which colour each screen pixel should be, the engine uses a texture filter.

The "nearest" texture filter simply takes the colour of the pixel in the texture that's closest to the position it's trying to render. This is why it's great for pixel art: when you're displaying your sprite at its native size (one sprite pixel = one screen pixel), "nearest" filtering keeps your pixels crisp and blocky.

However, if your camera moves smoothly, the position of your sprite on the screen can fall between screen pixels. Even if you're displaying your sprite at its native size, the engine needs to decide which screen pixels to colour in, and it can't perfectly align the sprite pixels with the screen pixels. This is when you can start to see jittering with "nearest" filtering: the engine suddenly shifts from one sprite pixel to another as the sprite moves across the screen.

The Solution

The way to solve this is by re-enabling linear texture filtering but apply a shader script like so:

shader_type canvas_item;

// Texture must have 'Linear filter' enabled!

// Automatic smoothing
// independent of geometry and perspective
vec4 texturePointSmooth(sampler2D smp, vec2 uv, vec2 pixel_size)
{
    vec2 ddx = dFdx(uv);
    vec2 ddy = dFdy(uv);
    vec2 lxy = sqrt(ddx * ddx + ddy * ddy);

    vec2 uv_pixels = uv / pixel_size;

    vec2 uv_pixels_floor = round(uv_pixels) - vec2(0.5f);
    vec2 uv_dxy_pixels = uv_pixels - uv_pixels_floor;

    uv_dxy_pixels = clamp((uv_dxy_pixels - vec2(0.5f)) * pixel_size / lxy + vec2(0.5f), 0.0f, 1.0f);

    uv = uv_pixels_floor * pixel_size;

    return textureGrad(smp, uv + uv_dxy_pixels * pixel_size, ddx, ddy);
}

void fragment()
{
    COLOR = texturePointSmooth(TEXTURE, UV, TEXTURE_PIXEL_SIZE);
}

As someone who builds pixelart games, this shader would need to be manually applied to every single node individually that uses pixelart textures.

Describe the feature / enhancement and how it helps to overcome the problem or limitation

Introduce an easier mechanism to avoid jittering for pixelart games. On the contrary to https://github.com/godotengine/godot-proposals/issues/5658 this proposal is suggesting a new option in the project settings to apply the Linear Filter + Shader to every texture.

Describe how your proposal will work, with code, pseudo-code, mock-ups, and/or diagrams

Approach A: New Texture Filter Sharp Bilinear

Introduce a new texture filter option in the settings that is effectively the linear filter plus the shader code mentioned above. One of the drawbacks here is that technically this is not a traditional texture filter but a post processing effect.

Approach B: Default shader preset for textures

Rather than baking this pixel-perfect shader into the engine, Godot could provide a mechanism to define the shader once in the settings somewhere. Any newly created node using the textures will then have that shader applied by default. One drawback here is that it requires more setup/knowledge from a user. Also, the behaviuor might be undefined if I attempted to define a custom shader for a specific texture -> applying a 2nd shader pass by default on any texture may be unnecessarily costly.

Approach C: checkbox in project settings

Introduce a new checkbox/toggle in the Texture project settings. It can only be checked in case Nearest filter is not enabled and it could apply the shader mentioned above in case this checkbox is toggled on.

Approach D: screenspace shader

Another option could be not to apply the shader per texture but once as a final pass on a Viewport. The drawback here is that it would require a custom viewport, as every single pixel on the screen needs to have the same size at all times (relates to https://github.com/godotengine/godot-proposals/issues/6389)

If this enhancement will not be used often, can it be worked around with a few lines of script?

As explained in this proposal, there is a solution already but it is not very scalable nor user-friendly, as a custom ShaderMaterial has to be attached to every single node that uses pixelart textures.

Is there a reason why this should be core and not an add-on in the asset library?

This is a very generic usecase affecting every person building pixelart games with smooth camera scrolling. A lot of people are unaware of how to fix the jittering to begin with, causing them to simply use Nearest filter.

[Calinou]

• Related to https://github.com/godotengine/godot-proposals/issues/96, as this involves adding more shader code to CanvasItemMaterial (with all the potential performance caveats of doing so).

Another concern is that using a sharp bilinear shader on every 2D node likely has greater performance demands compared to only using it once on the final Viewport image. While this probably won't be an issue on desktop, it can be on mobile/web platforms, especially at higher resolutions.

[bsil78]

I remember Solo CodeNet, as devlog of his Maggy game (https://youtu.be/UM9VOIn_WEU), presented something like the viewport solution with micro compensations trick approach in 3.5 or 3.4. linked to camera movements. Could it be related ?

[lostminds]

I remember Solo CodeNet, as devlog of his Maggy game (https://youtu.be/UM9VOIn_WEU), presented something like the viewport solution with micro compensations trick approach in 3.5 or 3.4. linked to camera movements. Could it be related ?

I think this is a very good and elegant solution for smooth camera movements without any additional shaders. But it looks like it requires a little work to set up. So perhaps making it easier to make this kind of camera rounding fraction-offset viewport would be a good idea, or even adding some such automatic offset option to the main viewport for pixel art games?

[golddotasksquestions]

Have you considered sharing this shader on the Asset Library? If you are using this for your game, I think it would also be a interesting devlog to talk about and would bring more attention to your issue.

Built-in postprocess shader stack is something I have wanted and asked for along time. If we would have such a built-in shader stack, I tink this shader would be a really nice addition.

[Zireael07]

There is a recent proposal or PR for built-in postprocessing

[golddotasksquestions]

@Zireael07 Do you mean https://github.com/godotengine/godot-proposals/issues/7916? As far as I understood, this is not really a stack of built-in post processing shaders which would also allow adding custom shaders to that stack. But I might be wrong, I have trouble following the discussion.

[Dusk-Dawn]

This is exactly my prefered method and the default for the Hi-res pixelart games. I really don't understand how that's not a setting somewhere when is something so fundamental for pixelart games. Does anyone know a scalable solution, like applying @bitbrain shader to "something" in autoload ?

[Lexpeartha]

Another concern is that using a sharp bilinear shader on every 2D node likely has greater performance demands compared to only using it once on the final Viewport image. While this probably won't be an issue on desktop, it can be on mobile/web platforms, especially at higher resolutions.

What about introduction of something like autoload but for shaders? Something where you can specify on which node types it is actually applied on (In this case it could be Sprite2D, AnimatedSprite2D etc) and there could be an option in project settings to register this specific shader to these types of nodes.

I think this somewhat alleviates performance concerns, since not every node will have this shader.

[Calinou]

What about introduction of something like autoload but for shaders?

This is being tracked in https://github.com/godotengine/godot-proposals/issues/8366.

[KeyboardDanni]

You probably don't want to apply all the shaders to each individual sprite in the game. Aside from the added complexity, I don't think it would give the effect you want.

Personally I'd really like a way to use a custom shader for when the root viewport is drawn to the screen. Especially since it's already handling things like aspect ratio for you. You can jerryrig a setup with your own viewports but this adds a lot of boilerplate and makes scene switching more complicated than it should be. Root viewport shaders would make it easy to apply not only sharp bilinear but other shaders like scanlines and CRT simulation.