Performance improvements - different DXT compression algorithms?

[DouglasHeriot]

One issue I have consistently as a media server user is convincing my content creation teams that the long export times of HAP codec is worthwhile. I've done some basic testing of different HAP encoders (QT Plugin, FFMPEG, and this Adobe Plugin), and found that FFMPEG runs significantly faster.

Here's my very quick, non-scientific, ball-park test results converting a 6 minute long video from ProRes 422 LT to HAP: (2015 MacBook Pro, 2.2GHz i7, Quad Core with hyperthreading)

Encoder	Time	CPU Usage
HAP Adobe Plugin 1.0.8-rc4 (Media Encoder 15.1.1)	14m 49s	All CPU cores used
HAP QT Plugin (Media Encoder 15.0)	20m 16s
AVF Batch Converter (highest quality)	22m 30s	All CPU cores used
AVF Batch Converter (lowest quality)	1m 46s	All CPU cores used
FFMPEG 4.0.2	3m 36s	Only single core used

It's interesting what the quality slider is doing in the AVFoundation implementation, and interesting that FFMPEG beats the other encoders by so much, and only uses a single core!

Doing some CPU profiling on my Mac with Xcode Instruments, reveals that almost all of the time in this Adobe plugin is spent inside the DXT compressor (Squish).

I found that the FFMPEG project uses a different DXT compressor instead of Squish: https://github.com/FFmpeg/FFmpeg/blob/a0ac49e38ee1d1011c394d7be67d0f08b2281526/libavcodec/texturedspenc.c

And it looks like the other Vidvox implementations in QT and AVF use either Squish, or GPU-based DXT compression for lower quality but higher speed. https://github.com/Vidvox/hap-in-avfoundation/blob/master/source/DXT/HapCodecGL.c

After a bit of Googling, I found that Id software wrote a paper in 2006 about Real-Time DXT compression, and then Intel in 2012 improved upon this with more modern SSE instructions. https://software.intel.com/en-us/articles/fast-cpu-dxt-compression The Id paper shows a 50x speed increase over Squish. This sounds possible, considering FFMPEG with single core was 8x faster than Squish (this Adobe plugin) which used 8 cores.

So what I'm wondering now:

• How much have you already investigated and tested different DXT compression methods? (I'm pretty new to learning about this)
• If I found the time (no promises!), would you accept a pull request that introduced a "fast but lower quality" option that used either this Intel or FFMPEG's DXT compression algorithms?
• Is anyone else out there already working on this?

[DouglasHeriot]

Ha, now I feel silly. Just saw PR #4 Thanks @exscriber I guess I just learned a bit more about HAP and DXT compression, and will have to try out the latest head build.

[GregBakker]

Those are great questions, and thanks for the investigation!

I haven't personally looked into 'fast' compares to ffmpeg's compression; would like to find out. Tom Butterworth (@bangnoise) probably knows! So I don't have benchmarks of how our 'fast' will compare either.

I'd like to do a new release with the quality settings ASAP, but want to address some of the installation issues first. It's certainly good to know they're going to be used though!

[DouglasHeriot]

Yes, the fast options are definitely going to get used! When doing previz in Disguise I just want our content creators to get me draft proxy files as soon and painlessly as possible, so I'll definitely recommend whatever the fastest encode settings are for that. Then also during shows when we're making last-minute changes after doors have opened or even exporting new files during a show we'll take the fastest encode (at reasonable resolution) over quality for sure!

[exscriber]

HapQ also have very fast encoding algorithm with high image quality, but 2x filesize.

Other thing for improve encoding time: get rid of post-render copy step: https://forums.adobe.com/message/10559705#10559705 If it ever possible...

[GregBakker]

re: post-render copy step - I'll create a separate issue for that, it's a good point.

Let's keep this issue open to discuss Hap-codec-specific encoding speed improvements.

[tromerk]

I am not knowledgeable on what compressor is being used in which encoder. And of course we all like it to be fast. But not at the expense of quality.

My previous findings (this was more than a year ago - not sure if anything has changed) were that the quality results while using FFMPEG were not as good as when using AME (prior to recent releases): https://github.com/Vidvox/hap-qt-codec/issues/39

[ttoinou]

Hello

I have done my share of testing and yes for certain kind of content, the ffmpeg algorithm has a lower quality even if much faster

It feels like for most content out there this algorithm is fine though.

For the Id paper : my tests indicate that it is what's used for fast compression in Hap Q (YCbCr mode). If you use this for RGB (Hap / Hap Alpha) then the quality is very low, ugly and blocky (but very very fast !). Could be useful for fast previews if your source content is generated also in a fast way

[bangnoise]

Just to confirm what it sounds like you have all already discovered - yes there are faster compressors, yes they are lower quality. Any pull request which speeds up compression whilst maintaining quality would be very welcome - preferably backed up by MS-SSIM and visual comparisons for a reasonable range of test materials.

[GregBakker]

Agreed, with that in mind I've raised #23

[ttoinou]

Looking at improving gradients and fixing Hap Q color shift also would be great..

[GregBakker]

FYI the post-export copy step was worked around a while back. Was in the 1.2.0-alpha3 [and likely earlier builds, would need to check back].

GPU-based encoding would likely be faster, so this issue is still relevant.