SKSL does not allow SamplingOption Mode "Linear" for Interpolation

najak3d commented 10 months ago

Description

I've got SKSL working well, except for that I can't seem to find a way to make SKSL 'sample(shader, uvCoord)' to 'interpolate'.

From the Skia online docs, they advertise a method for creating the child effects, for sampling another surface, like this:

sk_sp<SkShader> imageShader = image->makeShader(SkSamplingOptions(SkFilterMode::kLinear));

But SkiaSharp 2.88.xx, doesn't expose 'SKFilterMode' nor 'SKSamplingOptions' types/members.

Therefore, it looks like 'sample' in the SKSL always just does 'GetNearest (pixel)' producing a very blocky effect, without any interpolation.

Is SkiaSharp 3.0, going to expose this functionality? If so, then I'll switch to the alpha now, and give it a try.

Code

Just to do the job of what should have been a "single Sample" if the sampler with "LinearMode" turned on, instead of "closestPixel".

Here's what our code WOULD LOOK LIKE, if we could turn on LinearMode for sampling:

float3 elem = sample(elev_map, cf).xyz; // DONE!!  one line of code!

But since we're stuck with "ClosestPixel" sampling, our code instead looks like this:

float2 fc;
float fcxi = cf.x;   // raw fragCoord.x
float fcyi = cf.y;   // raw fragCoord.y
float fcxi0 = floor(fcxi);
float fcyi0 = floor(fcyi);

// These are 'lerps'.
float lpX = fcxi - fcxi0;
float lpY = fcyi - fcyi0;
float iLpX = 1.0 - lpX;
float iLpY = 1.0 - lpY;

float fcx0 = fcxi0;
float fcy0 = fcyi0;
fc = (float2(MinUV + fcx0, MinUV + fcy0));
float3 el00 = sample(elev_map, fc).xyz; // top left pixel
fc.x += 1;
float3 el10 = sample(elev_map, fc).xyz; // top right pixel
fc.y += 1;
float3 el11 = sample(elev_map, fc).xyz; // bottom Right pixel
fc.x -= 1;
float3 el01 = sample(elev_map, fc).xyz; // bottom Left pixel

// Now combine the 4 samples with lerp math applied (straight linear blending)
elevNorm = (el00 * iLpX * iLpY) + (el10 * lpX * iLpY) + (el01 * iLpX * lpY) + (el11 * lpX * lpY);

NEXT -- we have to do this for TWO input textures.... so we're doing a whole lot more code/work to get something very simple done.

It would be AWESOME if we could have a fix for this ASAP, rather than wait for 3.0.0, which is still in alpha.

Expected Behavior

I'd really like to see SKSL 'sample' method allow 'interpolation' rather than just doing 'GetNearest(pixel)', which produces a blocky effect for us.

Actual Behavior

No option for telling SKSL to 'interpolate' when sampling another surface.

Version of SkiaSharp

2.88.3 (Current)

Last Known Good Version of SkiaSharp

Other (Please indicate in the description)

IDE / Editor

Visual Studio (Windows)

Platform / Operating System

Windows

Platform / Operating System Version

No response

Devices

No response

Relevant Screenshots

Here's the the blocky "Relief shading" for our App:

Here's a view of the same data, and equivalent OpenGL shader, but using interpolation: (this is what we want to achieve with SKSL)

This 2nd snapshot also shows "contour lines" achieved with 'fwidth' function in OpenGL, which we also hope is supported by SkiaSharp 3.x.

Note -- here's the result now of a simple Image Blit, using a single sample (Nearest Pixel mode):

Here's what this should look like instead:

Relevant Log Output

No response

Code of Conduct

[X] I agree to follow this project's Code of Conduct

najak3d commented 10 months ago

Turns out that this issue is the worst issue plaguing us when using SKSL shaders. I can't even do a simple "Scaled image blit" shader without having to MANUALLY sample the texture 4 times, then do the Lerp Math to do blending/etc, to make this work reasonably well.

Now have a shader that is mixing two input bitmaps, both scaled (by different amounts) -- which means that we're now sampling each texture 4 times (8 samples total), and then having to do the Lerp math between 4 points twice as well...

Just to do the job of what should have been a "single Sample" if the sampler with "LinearMode" turned on, instead of "closestPixel".

Here's what our code WOULD LOOK LIKE, if we could turn on LinearMode for sampling:

float3 elem = sample(elev_map, cf).xyz; // DONE!!  one line of code!

But since we're stuck with "ClosestPixel" sampling, our code instead looks like this:

float2 fc;
float fcxi = cf.x;   // raw fragCoord.x
float fcyi = cf.y;   // raw fragCoord.y
float fcxi0 = floor(fcxi);
float fcyi0 = floor(fcyi);

// These are 'lerps'.
float lpX = fcxi - fcxi0;
float lpY = fcyi - fcyi0;
float iLpX = 1.0 - lpX;
float iLpY = 1.0 - lpY;

float fcx0 = fcxi0;
float fcy0 = fcyi0;
fc = (float2(MinUV + fcx0, MinUV + fcy0));
float3 el00 = sample(elev_map, fc).xyz; // top left pixel
fc.x += 1;
float3 el10 = sample(elev_map, fc).xyz; // top right pixel
fc.y += 1;
float3 el11 = sample(elev_map, fc).xyz; // bottom Right pixel
fc.x -= 1;
float3 el01 = sample(elev_map, fc).xyz; // bottom Left pixel

// Now combine the 4 samples with lerp math applied (straight linear blending)
elevNorm = (el00 * iLpX * iLpY) + (el10 * lpX * iLpY) + (el01 * iLpX * lpY) + (el11 * lpX * lpY);

najak3d commented 6 months ago

NOTE - I am now testing out Skia 3.0.0-preview2.1 - and am still not seeing an option to set Sample Mode to LINEAR.

Is a fix for this planned for 3.0.0? This is one of the most important issues we now face for our custom shaders. We're continuing to read 4-pixels, and do all of this math inside the shader, as shown in my sample code above.

mono / SkiaSharp