Blending of PMA with non PMA

[Fraggle]

Hello Daniel,

We were having issues when we used some blend modes, especially additive with non PMA textures from ATF.

Example: The background on the left side is a PNG (so PMA) and on the right side an ATF (so non PMA) with blend mode none. The 2 green spots at the top are PNG (so PMA) in additive. The 2 green spots at the bottom are ATF (so non PMA) in additive. There are also a white cloud PNG between the two top spots and a white cloud ATF between the two bottom spots in additive. In the middle there is a render texture with only the alpha channel.

You can see that there is an issue where the alpha is not zero for the ATFs. It also appear when blendMode is Normal for non PMA on top of PMA.

We found out that the issue happen because the blend mode factors only take into account the source PMA state and expect the destination to be of the same PMA state. As we cannot know the destination PMA state (it can even vary from pixel to pixel), we came up with 2 solutions that would fix the bug.

The first solution is already discussed in another issue, we could alter the quadbatch fragment shader to alter the output color when not PMA to premultiply everything or the other way around, just to make sure everything is the same. Of course this is not without a potential performance impact since we add a MUL or a DIV in the shader for every pixel drawn from certain texture.

The other solution is simply to premultiply the source image supplied to the ATF converter and allow us to say wether the ATF is PMA or not (as right now you force the PMA flag to off when uploading an ATF). That way everything would be PMA and it will works flawlessly.

So my request would be for you to add a pma flag to TextureOptions and an additional argument to fromAtfData(..) plus the additional plumbing between AssetManager and Texture. It is transparent for existing users and it would fix all our issues with PMA / non PMA.

Here is the same example where we did just that : manually premultiplied the png before submitting to png2atf then forced the pma to true in fromAtfData().

I can do the patch myself and submit it if you want. What do you think ?

Seb

[Fraggle]

Btw, we just noticed that you convert the output to PMA in all filters (like what we proposed for solution 1). I still think we should implements solution 2 for performance reason but you can see that you already noticed that all your shader should output the same format.

[hejiacong]

I find there is no way to create an ATF with PMA.Is it possible?

[PrimaryFeather]

You're right, the PMA stuff is really painful ... I know that there are some problem with blend modes, as you encountered yourself, but I never saw a good solution for this. Undoing the premultiplication (or multiplying everything) was something I considered it too dangerous performance-wise (especially old hardware is very sensitive when it comes to fragment shader operations), and there are some blend modes that work only with PMA.

But the alternative solution sounds like a good idea to me, actually! May I ask how you premultiplied the PNG file — I guess with a custom script?

[b005t3r]

@Fraggle, this is actually quite brilliant :) I'd also like to know how did you premultiply the PNG and if using a custom tool, could you share it?

[Fraggle]

In that particular case we used texture packer "premultiply alpha" checkbox :-) But by quickly googling it seems that you can do it using imagemagick: http://imagemagick.org/discourse-server/viewtopic.php?t=23463 Or python, or c: https://github.com/maxme/PNG-Alpha-Premultiplier

Daniel, do you want me to provide a patch or you want to do it yourself ?

[PrimaryFeather]

Ah, damn, I totally forgot about that TexturePacker option. Makes sense! If you could provide a pull request, that would be great! I agree that this should be added, as it's a rather simple solution for those PMA problems.

[Fraggle]

Should be delivered during the afternoon :-) -edit- Well maybe tomorrow :)

[Fraggle]

Btw, I found this excellent article about PMA (why it's actually "the right format").

http://home.comcast.net/%7Etom_forsyth/blog.wiki.html#[[Premultiplied%20alpha]]

[IPv6]

@Fraggle nice tech blog! very interesting

[Fraggle]

Friends, I'm sorry to tell you that this solution actually do not work because of DXT format. Unfortunately, the dxt compression work on "squares" for the RGB channels and avoid artifacts by blending with the alpha channel (much less compressed).

In my opinion, the only way to have a reliable blending is to do it at the shader level.. I'm gonna do it in our fork of starling and propose the patch but I can understand that Daniel will want to avoid the extra call (I'll make it opt-in).

[PrimaryFeather]

Damn, that's a bummer. :disappointed:

Nevertheless, thanks a lot for your efforts! At least we now know that this is not an option. I'll consider making this change at the shader level ... having all textures be equal in terms of PMA would be great, and modern GPUs are probably not that sensitive about an additional fragment shader instruction.

[kheftel-old]

His followup article is also super in depth and has some great points: http://home.comcast.net/~tom_forsyth/blog.wiki.html#%5B%5BPremultiplied%20alpha%20part%202%5D%5D

Thanks,

Kawika Heftel Heftel Studios 801-358-9830 http://www.heftelstudios.com

[PrimaryFeather]

Thanks, Kawika! I'll give both articles a thorough read. (I already knew the first one, but it's been a while since I read it the last time ...)

[kheftel-old]

thanks, and good luck to you!! Glad we've got you and your skills working on this confusing part of GPU programming :)

[PrimaryFeather]

This thread was one of the main reasons why I decided to force PMA in Starling 2 for all rendering. On texture sampling, the fragment filters will automatically convert non-PMA textures to PMA. That way, this problem should finally be disposed once and for all. :sweat_smile: