gltfpack: Rebalance texture quality table

[zeux]

The main goal of this change is to tweak the UASTC RDO curve. The initial values here have been picked for early UASTC versions that used a different RDO algorithm, and haven't been revisited since.

The new table makes it so that at tq 5, we use the same defaults as basisu (q=128 for ETC1 and l=1 for UASTC RDO), and tweak the table to be in general a little more aggressive for UASTC. We use different curves before l=1 and after as the expectation is that for earlier quality levels, the size is much more important than quality.

The default lambda for tq=8 is a little larger now (0.4 vs 0.3) which will result in a small size improvement and still good quality by default. There's maybe a little more room for default lambda to be higher, but eg lambda=1 does not seem practical for normal maps on a range of production assets, so we keep the curve for tq=6..10 to be more quality focused.

The new table is not based on a rigorous fitting of the actual quality/size curves, but the new tool gltfbasis.py can help find a more optimal table in the future.

[zeux]

Adding an output of gltfbasis.py that was used to try to understand the reasonable values of RDO lambda using a variety of textures from glTF-Sample-Assets:

Unfortunately, PSNR and RMS aren't always a great indication of whether or not the quality is acceptable. For example, when using (after this change) -tq 5 to compress Corset scene, which uses lambda 1, I get fairly objectionable artifacts on the normal map, which is due to some blocks being flattened out entirely:

... and yet looking at the quality curves it would be hard to convincingly say "lambda 1 is bad for this texture".

So the new table is still fairly hand wavy and another tuning attempt might be necessary in the future. Also the "early" part of the table (tq < 5) may not be super practical for UASTC, but my recollection is that low quality levels for ETC are also really not great even for color, so from that perspective tq < 5 is explicitly biased to minimize the size at all costs.

[zeux]

There might be a benefit to selecting different lambda (maybe with scaling?) per quality level, because for color textures a higher lambda is still reasonable. It's a little difficult to differentiate between normal and attrib in the same way: some assets have roughness maps that are fairly tolerant to compression artifacts, but some have ones that suffer fairly significantly. Still, maybe it would be reasonable to increase lambda a little for attrib, and even more for color, in the future automatically.

This could be done separately with a kind-based scale (I think lambda is reasonably tuned now for normals at least...), so maybe will happen in a separate change, but there's also always a possibility to set kind-specific quality like this:

-tc color -tu normal,attrib -tq attrib 5

This will use ETC1S for color textures, UASTC for normal maps with default quality 8, and UASTC for metal/roughness/etc maps with lower quality 5 (lambda=1).