ASTC Feature Proposal / vs ETC2 Discussion

[fire]

After BC1, BC7 etc, after all the ones that are important, I would petition for supporting ASTC with compute shaders.

https://www.khronos.org/opengl/wiki/ASTC_Texture_Compression

https://github.com/ARM-software/astc-encoder appears to be the standard encoder, but I don't know if there's other ones.

https://github.com/GameTechDev/ISPCTextureCompressor and https://github.com/GPUOpen-Tools/compressonator are the other tools in this space that also support astc.

ETC2 has significant banding issues for the mobile hardware and is noticeable on the Oculus Quest.

I appreciate your work on this! Your work is amazing.

[darksylinc]

Hi!

ASTC is low on the priority list for now, so I don't want to keep your hopes up. Maybe one day :)

That being said, I read one thing that surprised me:

ETC2 has significant banding issues for the mobile hardware and is noticeable on the Oculus Quest.

Weird. ETC2 usually has comparable quality to ASTC. It is possible the encoder you're using is actually encoding ETC1 rather than ETC2 which would explain it.

ETC1 is considerably lower quality than ETC2, but because ETC2 is backwards compatible (ETC1 textures can be fed to ETC2 decoders) often etc1 is labelled as 'etc2' but the encoder didn't actually use the quality enhancements.

What encoder are you using for etc2?

At the time being betsy only supports ETC1 output, but it may support ETC2 in the future.

But more importantly, banding problems can be often fixed using dithering! Did you try using dithering? Betsy does support it and can be used via --dither argument

In some cases it does more harm than good, and not all textures are suitable for dithering. But when it does, it looks like magic.

I appreciate your work on this! Your work is amazing.

Thanks!

[fire]

We were using the https://github.com/GPUOpen-Tools/compressonator encoder through gui.

We used the image diff tool to compare a texture sheet's conversion to etc2 and astc.

Trying to get you the sample image, but don't have it yet.

[lyuma]

Hi, Fire told me about this tool and I am eager to use it. Really excited to see this GPU compression implementation using shaders. Sorry for the late reply, but just a followup on ETC2 and the quality loss I see:

Here is the source texture atlas I've been using to test, for a 3d character model:

Here is the diff between the original and the output ETC2 image: you can see heavy color banding in some parts, as well as the wrong color being used: The banding in the face especially is noticeable.

The --dither option added recently appeared to give an error, so I was unable to test it:

../betsy ~/Downloads/texatlas_albedo.png --codec=etc2_rgb --quality=2 --dither etc2_rgb_dither.ktx
Initializing API
GL Context creation suceeded.
Error opening ../Data/dither555.glsl
Shutting down

By the way, etc1 behaved identically to etc2 and actually produced output which was indistinguishable... in fact, unity claims the etc1 generated image is in "RGB Compressed ETC2" format, so it is possible there is a mixup, but I am sure I used --etc1

While I'm open to trying dithering, I think there is a case to be made for ASTC if you need color accuracy without dithering.

Thanks, Lyuma

[darksylinc]

Hi

The --dither option added recently appeared to give an error, so I was unable to test it:

It seem you called the file from the wrong folder. You need to run it from the bin/Release (or Debug) as it will look for "../Data" (a relative path)

Betsy at the moment only supports etc1. I tried it and indeed has heavy banding. I tried dithering but it just made it worse.

By the way, etc1 behaved identically to etc2 and actually produced output which was indistinguishable... in fact, unity claims the etc1 generated image is in "RGB Compressed ETC2" format, so it is possible there is a mixup, but I am sure I used --etc1

Yeah, because of the backwards compatibility of ETC2, Khronos (the entity ruling over OpenGL & Vulkan) decided to not include an "ETC1" flag thus all etc1 textures will be flagged as etc2.

It is fishy that there is no visual difference between etc2 & etc1. The banding we're seeing is caused by slowly-varying colour, which is in theory handled by the special "P mode" or "Planar Mode" added in etc2.

Thanks for the source image. I'll see if there's something that can be done with etc1/2

[lyuma]

I double-checked, but I suspect maybe dither555.glsl wasn't committed... it was able to find the other glsl files. (I see the file referenced from EncoderETC1.cpp line 110, but I can't find that file in git)

I noticed that the code appears to do the same thing regardless of whether --etc1 or --etc2_rgb are passed

switch( params.codec )
{
    case Codec::etc1:
    case Codec::etc2_rgb:
    case Codec::etc2_rgba:
...

Also, I did test etc2_rgba but there was no difference of the RGB channels compared to etc1 or etc2_rgb.

Thanks!

[darksylinc]

Hi!

I just finished implementing etc2_rgb in betsy. I think I still need to clean some stuff tomorrow so it's possible not all options are working (e.g. etc2_rgba still uses etc1 for the RGB parts)

There IS a very noticeable quality difference.

This is betsy with etc1_rgb:

This is betsy with etc2_rgb:

[lyuma]

Nice work! It's looking way better. I still see some issues in specific parts of the image: ... but many of the noticeable artifacts are gone.

I also tried with the Alicia VRM texture sheets (Alicia is open licensed and part of the VRM model spec). Again much improved, still a few artifacts. I'll attach the two original textures here.
anyway here are the diffs of the above. Overall much better, though with some parts that have color inaccuracy...:

Side note... I had trouble getting the resulting KTX textures exported to PNG and had doing this without gamma or banding artifacts that arise from improperly converting sRGB. The above should be correct, but if you happen to know of a good way to convert the generated KTX textures back to PNG, I'd actually find that helpful when testing...

Once again, thanks for all your work on this!

[fire]

@darksylinc Do you think you'll be able to work on these improvements? Curious where this is at.

Also it would help wider testing if there were testing binary releases for Win10 and Linux. Mac is less used, so it's up to you.

[darksylinc]

Hi!

@darksylinc Do you think you'll be able to work on these improvements? Curious where this is

I doubt it. When you first posted, ETC2 had not yet been implemented and had a good chance at solving the issues you were describing, particularly the 'P mode' introduced in ETC2 which deals with slowly-varying gradients and encodes colour with more bits. This makes it ideal to hand-painted textures like the one you presented.

However ETC2 has now been implemented and this is 'the best it can offer'. Perhaps the encoder can be further improved at generating better quality but I ignore how that could be achieved.

I could also add parameters to interactively tweak which ETC mode to chose from rather than only relying on automatic selection via MSE (mean squared error) however this is a bit user hostile since users want something that just works automagically, rather than tweaking params for each texture individually until it looks right.

Like you originally suggested, ASTC may offer a quality improvement, but it is not within plans to support ASTC at least for now. Also beware ASTC 4x4 consumes twice the memory of ETC2, which is how it manages to look better. The compression ratio of ETC2 is somewhere between ASTC 6x5 and 6x6

Also it would help wider testing if there were testing binary releases for Win10 and Linux. Mac is less used, so it's up to you.

A binary release for Windows planned for next week. Perhaps I'll make a Linux binary release, but I don't want to deal with distro compatibility right now.

Take in mind betsy was very WIP up until a few weeks ago. It only stabilized up until recently.

[fire]

Thanks!