Using -pp-premultiply with sRGB compression gives a brigher than expected output

[ydcoming]

When I convert an image using the command-line tool astcenc with the command astcenc -cs k.png k@1.ktx 6x6 -medium -pp-premultiply, the resulting image appears overly bright when displayed in WebGL.。

However, when I manually handle the pre-multiplication of the Alpha channel using the sharp library in Node.js and then use the command-line tool astcenc with the command astcenc -cs k.png k@1.ktx 6x6 -medium, the image displays correctly in WebGL.

codes are like this；

const sharp = require('sharp');
const fs = require('fs').promises;
/// 预乘Alpha操作 png才能使用 所以在使用前先判断通道是否是四个
async function manualPremultiplyAlpha(inputPath, outputPath) {
  try {
    // 读取图像为Raw格式
    const { data, info } = await sharp(inputPath).raw().toBuffer({ resolveWithObject: true });

    // 基于图像的宽度、高度和通道数进行遍历
    const width = info.width;
    const height = info.height;
    const channels = info.channels;
    if (channels !== 4) {
      throw new Error('Input image must have 4 channels (RGBA).');
    }
    for (let y = 0; y < height; y++) {
      for (let x = 0; x < width; x++) {
        // Calculate the offset of the current pixel in the data array.
        const pixelOffset = (y * width + x) * channels;

        // 获取Alpha值并进行归一化（假设8位精度）
        const alpha = data[pixelOffset + 3] / 255;

        // 预乘Alpha操作
        for (let c = 0; c < 3; c++) {
          data[pixelOffset + c] = Math.round(data[pixelOffset + c] * alpha);
        }
      }
    }

   // 使用sharp处理预乘Alpha后的Buffer并输出为PNG
   await sharp(Buffer.from(data), { raw: { width, height, channels } })
   .png()
   .toFile(outputPath);

    console.log('Image processed and saved with manual pre-multiplied alpha.');
  } catch (error) {
    console.error('Error processing image:', error);
  }
}

// 使用函数处理图像
const inputFilePath = 'k.png';
const outputFilePath = 'output.png';
manualPremultiplyAlpha(inputFilePath, outputFilePath);

；

[ydcoming]

[solidpixel]

~The current code for premul doesn't do anything with color space information ...~

EDIT: Ignore, yes it does.

[solidpixel]

How are you loading your texture in the graphics API? I suspect you are loading it as a linear texture format, not an sRGB texture format, so the correct gamma curve is not being applied when the texture is being sampled.

[solidpixel]

Input values into the compressor:

• Before pre-mul: 1.0000 1.0000 1.0000 0.4941
• After pre-mul: 0.7315 0.7315 0.7315 0.4941

This looks correct to me for an sRGB input image. The 0.7315 in sRGB gamma = 0.4941 in linear gamma.

I strongly suspect you are reading this as a linear texture in your shader, so missing the required gamma correction for an sRGB input. For linear textures you should be using -cl not -cs.

[ydcoming]

What are the differences between processing premultiplied alpha using the Sharp library versus Ascent? Although both are supposed to perform premultiplication of the alpha channel, the resulting outputs differ. Furthermore, even though the same API is invoked in both cases, why do different results occur? I am using the LayaAir engine.

[solidpixel]

No idea - I've never used Sharp or Ascent. However from my point of view, the compressor is currently doing the right thing for sRGB textures, which is applying the pre-multiplication in linear-space.

If you want the other behavior, which is pre-multiplication without sRGB color conversion, use -cl not -cs when compressing.

[ydcoming]

My apologies for the confusion in my description. What I meant was, does using the Sharp library for premultiplication, as shown in the code above, result in inconsistencies compared to the algorithm used within the compressor? Does the compressor only support linear premultiplication?

[solidpixel]

result in inconsistencies compared to the algorithm used within the compressor

Yes, the code you posted is ignoring color space information, so applying pre-mul in sRGB domain.

Does the compressor only support linear premultiplication?

If you want the other behavior, which is pre-multiplication without sRGB color conversion, use -cl not -cs when compressing.

[solidpixel]

Also, going back to to an earlier question ... Are you sure these are actually loaded as sRGB ASTC textures in the graphics API? What is the API format being used when you use these at runtime? Compressing with -cs only makes sense if you sample from the texture as an sRGB texture.

[ydcoming]

"Maybe I didn't describe clearly at first. Let me detail my steps now. I conducted a comparison. The first group of operations is as follows: I used the Sharp library for PremultiplyAlpha, with the code as mentioned earlier. Then I passed the processed image to the compressor with the parameters: astcenc -cs output.png k@1.ktx 6x6 -medium. The colors appear normal in the LayaAir engine.

The second group of operations is: I directly used the compressed PremultiplyAlpha with astcenc -cs k.png k@1.ktx 6x6 -medium -pp-premultiply. In the LayaAir engine, the brightness is higher. What I mean is, there's a difference between how I handled PremultiplyAlpha myself and how the compressor did it."

[ydcoming]

"I also consulted the official LayaAir engine team. They indeed use the Sharp library first for PremultiplyAlpha and then utilize pvrtextoolcli for texture compression."

[solidpixel]

Not sure how many ways I can say the same thing.

Stop using astcenc sRGB mode (-cs) in your command line and use linear mode (-cl). Does that work?

You are compressing this as an sRGB texture (-cs). How is the game engine setting up the texture? Is it using an sRGB or a linear format enum in the graphics API? They are different things, and need different handling for premul.

It only makes sense to use sRGB mode (-cs) in the compressor if the game engine is configured to read the data as an sRGB texture. It looks like the game engine is reading it as a linear texture, so you should be using linear mode (-cl) when compressing.

[ydcoming]

yes -cl not work it didn't work and it turned completely white.

[solidpixel]

In linear encoding it should look darker - the border goes to 0.4 with premul, instead of 0.7.

[solidpixel]

Original input image:

Output image after compressing with linear-space premultiplication:

./bin/astcenc-avx2 -tl in.png in-linear-pre.png 6x6 -medium -pp-premultiply

Output after reading as a linear texture and blending on to a 136/255 grey background, which gives a 149/255 output which matches your "good" example above (modulo rounding):

./Utils/astc_blend_test in-linear-pre.png bt-out-l.png linear pre off

[solidpixel]

This looks like a colorspace issue in your content pipeline and/or application - everything in the compressor seems to working correctly.

Unless you can point at something specific the compressor is doing wrong, I'm going to close this ticket as the issue is not related to the to compressor behavior.