Bitmap blit blending algorithm (bitmapBlit.frag) is not 100% accurate for color values.

[Ancurio]

According to my personal tests, the resulting alpha value is correct for all inputs (srcAlpha, dstAlpha, bltAlpha*), however, the color value is just an approximation as I have yet to successfully reverse-engineer the algorithm behind it. The approximation is written to favor text display (with eg. manual shadows, or at half opacity) as much as possible, but many visible irregularities remain and the code itself is quite ugly. Note that in case of blitting with bltAlpha = 1.0 on a cleared surface (the vanilla way text is displayed in RMXP games), no blending is necessary and the source is directly copied into the destination (ie. a literal blit).

The color value has 4 inputs: srcColor, dstColor, srcAlpha*bltAlpha and dstAlpha, The results get especially confusing once the alpha values start deviating from 1.0.

(*bltAlpha is the 'opacity' parameter passed into the blt functions, or the alpha value of the font used when drawing text).

[cremno]

Could you give this a try?

resFrag.rgb = as*srcFrag.rgb/resFrag.a + (1.0-at)*ad*dstFrag.rgb/resFrag.a;

It (hopefully) does the same as the HLSL shader I've written a few years ago. I think the results were fine back then but I've never (extensively) tested it.

[Ancurio]

Hey, that looks awesome! I haven't checked this against the "value dump + gnuplot" method I used before, but from a quick look with two test projects, the results are amazing. How did you find this out? I skimmed a couple GDI docs from Microsoft, but couldn't find anything useful.

Do you want to make a pull request (so it has your name on it)?

[Ancurio]

Actually, don't make a pull request yet. I found that this equation leads to a bit of a strange result when used with images (and not fonts):

RMXP: Yours:

(This is at half opacity)

Your equation still looks superior Font rendering in general; maybe it's time to split off the text drawing and the bitmap blitting path into two shaders? Anyway, my right hand is a bit strained at the moment so I can't type a lot, but I intend to conduct a couple more in-depth tests later.

[cremno]

Um, okay. That looks like resFrag.a is not right but according to your tests it is, isn't it? I used a different formula for this one. And I'm going to install mkxp's dependencies in a few minutes and test it myself.

And the RGSS is most likely not using GDI/DDraw in this case.

[cremno]

diff --git a/shader/bitmapBlit.frag b/shader/bitmapBlit.frag
index 1913c5b..c50e0b2 100644
--- a/shader/bitmapBlit.frag
+++ b/shader/bitmapBlit.frag
@@ -25,13 +25,13 @@ void main()
        float ad = dstFrag.a;

        float at = ab*as;
-       resFrag.a = at + ad - ad*at;
+       float af = resFrag.a = at + ad*(1.0-at);

        // Sigh...
        if (ad == 0.0)
                resFrag.rgb = srcFrag.rgb;
        else
-               resFrag.rgb = as*srcFrag.rgb + (1.0-at) * ad * dstFrag.rgb;
+               resFrag.rgb = at*srcFrag.rgb/af + (1.0-at)*ad*dstFrag.rgb/af;

        gl_FragColor = resFrag;
 }

opacity = 0x7f

Well, that looks better. Could you test it more extensively? You seem to have already scripts and tools to do this. I don't.

[Ancurio]

Sorry for the delay; my right wrist recovered a little so I was able to code again.

Um, okay. That looks like resFrag.a is not right but according to your tests it is, isn't it? I used a different formula for this one. And I'm going to install mkxp's dependencies in a few minutes and test it myself.

My earlier tests were all manual (pulling multiple sliders and comparing values), and it seemed a pain in the ass to use that again, it was mostly useful to guess the equation. So I wrote up a real testing script that compares about 1.4k different value tuples of srcAlpha, dstAlpha and bltAlpha, and the result was [[0, 1320], [1, 11]], ie. 99.99% of alpha values are correct, and the rest have an error of 1. So yeah, resFrag.a is definitely correct. I will use the same method to gather statistics about the accuracy of computed color values soon.

Oh, and while doing that, I spotted a really nasty bug in my #get_pixel implementation, yay!

float af = resFrag.a = at + ad*(1.0-at);

Not sure if intentional, but that's the same equation as mine, just with ad factored out. So I'm really not sure why the sword sprite's colors look so oversaturated with your first equation.

And the RGSS is most likely not using GDI/DDraw in this case.

Okay, you're the expert =) I was just guessing based on similar blt and stretch_blt function names that can be found in GDI headers. But seeing that I couldn't find a shred of useful information in MS' GDI docs, it makes sense that RMXP must be rolling its own blending code.

By the way, you said you wrote an HLSL shader to emulate the blit blending; would you mind posting that? It would be interesting to read as a reference =D

[Ancurio]

Awesome news! I just ran the color tests, and with your last code suggestion, I get

[0, 67.77] 
[1, 31.78] 
[2, 0.43] 
[3, 0.02] 
[4, 0.0]

ie. with an error margin of 2, that is 99.98% accuracy. You really did nail the algorithm :D For comparison, this is how my original code scored:

[0, 21.85] 
[1, 2.73] 
[2, 2.15] 
[5, 2.12] 
[3, 2.11] 
[4, 2.0]
(rest almost evenly scattered)

This is the code I ended up using:

if (resFrag.a == 0.0)
    resFrag.rgb = srcFrag.rgb;
else
    resFrag.rgb = (at*srcFrag.rgb + (1.0-at)*ad*dstFrag.rgb) / resFrag.a;

I test for resFrag.a == 0 instead because that's what really causes the couple NaNs.

I see you replaced as*srcFrag.rgb with at*srcFrag.rgb in your second code snipped (compared to your first), was the first one a typo?

[cremno]

That's great! Yeah, that was a typo and good to know that my alpha formula doesn't really differ from yours. I've changed it because I mistakenly thought it was the culprit.

And I'm going to post my RGSS clone this weekend. I've started re-organizing and updating it (to VS 2013 and DX 11) yesterday. But it isn't nearly as complete as yours. I've lost the interest back then and also I'm not really an expert in graphics programming (and especially maths :3). But it's an interesting topic.

[Ancurio]

Thanks again for your help! Looking forward to seeing your code.