S2TC_REFINE_COLORS=ALWAYS creates artifacts on transparent borders

[ghost]

I noticed very ugly texture borders when running the default settings of the library from 20121227 (seen in s2tc_refine_always.png). It doesn't happen with other S3TC implementations and when using S2TC_REFINE_COLORS=NEVER (with quite bad image color results; can be seen in s2tc_refine_never.png). The texture (input_texture.png) seems to be special because it has transparent parts which are not shown by default (texture coordinates of the rectangle don't include this parts). But it seems the color refinement of S2TC moves these parts into the image (somehow?)

[ghost]

convert input.png input.tga
export S2TC_REFINE_COLORS=ALWAYS
s2tc_compress -i input.tga -o output.dds -t DXT5
convert output.dds output.png
s2tc_decompress -i output.dds -o output.tga

will result in following output (both graphicsmagic and s2tc_decompress looks really similar/are the same)

[divVerent]

I think I could track it down.

The issue was that for the regular color distance functions, libtxc_dxtn's "fast" initial color finding is on as it doesn't hurt much (just pick a min and max color, let refining handle the rest) instead of the color reduction method used normally (out of the 16 colors of a block, pick the 2 with shortest average distance to the others).

In this specific picture, the alpha-zero pixels have white as color, making a typical block contain in the RGB channels:

"dark blue' (1 pixel) "medium blue" (3 pixels) "white" (12 pixels)

But, the bug was that the "fast" initial color selection ignored alpha-zero pixels, so it picked "dark blue" and "medium blue" as the two base colors. S2TC encoding then mapped "white" to the "medium blue" too, and when refining, recalculated "medium blue" from the 15 pixels mapped to it - 12 of which are white.

I don't fully see yet why REFINE_LOOP didn't have the same issue. Most likely it did, and you just didn't get it because of the specifics of the input image and the selected color distance function (by default a squared euclidean distance in the color space, with extra weight on green).

Now it will in the same case pick "medium blue" and "white" as the two base colors, mapping all the transparent pixels to white and using a single color for the others. This is as good as it gets, as alpha-zero pixels can't usually be ignored (e.g. when using standard linear scaling like in OpenGL's texture2D(), the very image you are using would get a white halo when scaling it up and rendering with transparency like in the attached image below). Similarily, not caring for the color of these pixels may often yield good results, but in Xonotic there were a few cases where this really ended up bad, so I now keep them to the color of the input image, assuming that whoever made the image knew what they were doing.

[divVerent]

I understood now why S2TC_REFINE_COLORS=LOOP didn't trigger it:

The bug happened like this:

1. Init: Choose dark blue, medium blue.
2. Encode: assign all pixels to these (the transparent ones get medium blue); initial score is quite bad. (S2TC_REFINE_COLORS=NEVER would stop here.)
3. Refine: Recompute colors; this will choose dark blue, very bright blue. Refining by definition never worsens the score for the current encoding, when the color distance function fulfills the triangle inequality (which the default one does). (S2TC_REFINE_COLORS=ALWAYS would stop here.)
4. Encode: assign all pixels to these (the transparent ones get the very bright blue, the others get the dark blue); this improved the score.
5. Refine: Recompute colors; this will choose dark blue, white.
6. Encode: assign all pixels to these (the transparent ones will get the white, the others the dark blue); this improved the score.
7. Refine: Recompute colors; this will choose dark blue, white.
8. Encode: assign all pixels to these (the transparent ones will get the white, the others the dark blue); the score stayed the same. Done.

Now, "dark blue, white" will be chosen from the start, fixing the issue.

Now that I know exactly what happened, I will make a new release tag soon.