Some devices do not report AVC High10 capabilities even though playback is supported, causing decoder checks to fail

[MichaelRUSF]

Describe the bug

The hasDecoder function in MediaCodecCapabilitiesTest does not work as expected. It always returns false for: supportsAVCHigh10
supportsHevcMain10

It works correctly if you test: supportsHevcMain
supportsAV1Main10
*supportsAVCProfileMain

@lawadr Also, the getDecoderLevel which is based off of the hasDecoder does not work for: getHevcMain10Level *getAVCHigh10Level

It works correctly if you test: getHevcMainLevel *getAVCMainLevel

This is why H264 High10 videos will always transcode. Also, why 4k HEVC content gets remuxed instead of being transcoded on devices that only support 1080p. I'm sure there are other bugs, but these are the two that I could test.

*Not implemented

Logs

No response

Application version

0.16.5

Where did you install the app from?

None

Device information

~

Android version

~

Jellyfin server version

10.8.3

[lawadr]

You haven't listed your device. Maybe it doesn't support those profiles?

[MichaelRUSF]

I tested out the code on Andriod Studio with the emulator. Lots of ways to test it, but I'll try to explain my method.

If this is true (MediaCodecCapabilitiesTest)

fun supportsAVCHigh10(): Boolean = hasDecoder(
        MediaFormat.MIMETYPE_VIDEO_AVC,
        CodecProfileLevel.AVCProfileHigh10,
        CodecProfileLevel.AVCLevel4
    )

Then because of this (ProfileHelper) AVC High 10 is supposed to direct play given no other factors, e.g., subtitles, level support, bitrate, etc..

    val h264VideoProfileCondition by lazy {
        ProfileCondition(
            ProfileConditionType.EqualsAny,
            ProfileConditionValue.VideoProfile,
            listOfNotNull(
                "high",
                "main",
                "baseline",
                "constrained baseline",
                if (MediaTest.supportsAVCHigh10()) "high 10" else null
            ).joinToString("|")
        )
    }

Using current build, play AVC High 10 L4.2 media, no subtitles or audio to eliminate other factors. Notice the video will transcode, reason should be unsupported profile.

Change this

fun supportsAVCHigh10(): Boolean = hasDecoder(
        MediaFormat.MIMETYPE_VIDEO_AVC,
        CodecProfileLevel.AVCProfileHigh10,
        CodecProfileLevel.AVCLevel4
    )

to this

fun supportsAVCHigh10(): Boolean = hasDecoder(
        MediaFormat.MIMETYPE_VIDEO_AVC,
        CodecProfileLevel.AVCProfileMain,
        CodecProfileLevel.AVCLevel4
    )

Run a new debug build on emulator and notice the file should now direct play without transcoding. HEVC Main will also come back as true. HEVC Main 10 comes back as false causing the video to transcode. You can run similar tests with different conditions to verify that getHevcMain10Level always comes back false with the getDecoderLevel function.

Note I also tested out the debug builds on real hardware and had the same results

[lawadr]

This is because the Android emulator doesn't claim support for HEVC Main 10 or AVC High 10:

Media type 'video/avc':
  Decoder "OMX.google.h264.decoder" supports
    profile/levels: [
      65536/65536 (ConstrainedBaseline/5.2),
          1/65536 (Baseline/5.2),
          2/65536 (Main/5.2),
      524288/65536 (ConstrainedHigh/5.2),
          8/65536 (High/5.2) ]
...
Media type 'video/hevc':
  Decoder "OMX.google.hevc.decoder" supports
    profile/levels: [
          1/65536 (Main/Main 5.1),
          4/65536 (MainStill/Main 5.1) ]

On this system, supportsAVCHigh10 and supportsHevcMain10 should both report false. I don't see what the problem is here. These functions seem to work as intended.

[MichaelRUSF]

Ok, thanks for that info. I didn’t realize that the emulators were not reporting AVCHigh10 & HEVCMain10 nor did it dawn on me to check. I assumed that they were b/c in my testing AVCHigh10 would directly play without transcoding in the emulator.

I also tested 2 Amazon devices; the Fire TV Stick 2nd gen and Fire TV Stick 4k 2nd gen, along with my Sony Bravia X90J. They all produced the results as the emulator returning false when checking for AVCHigh10 leading to transcoded video and True for HEVCMain & AVCMain producing direct play.

So my conclusions here are wrong. The main issue, as you pointed out, is that devices are not reporting AVCHigh10, even though they are supported. For example, Amazon's device specifically states “H.264. Hardware accelerated up to 3840x2160p (4K) @ 60 fps, 20 Mbps, High 10 Profile Level 5.2”. Yet reports no such decoder thus rendering the decoder check useless.

Decoder "c2.android.avc.decoder" supports
    aliases: [
      "OMX.google.h264.decoder" ]
    attributes: 0x4: [
      encoder: 0,
      vendor: 0,
      software-only: 1,
      hw-accelerated: 0 ]
    owner: "codec2::software"
    rank: 512
    profile/levels: [
      65536/65536 (ConstrainedBaseline/5.2),
          1/65536 (Baseline/5.2),
          2/65536 (Main/5.2),
      524288/65536 (ConstrainedHigh/5.2),
          8/65536 (High/5.2) ]
    colors: [

Well, this still is an issue but not with the hasDecoder function. It's more like what should be done when devices do not properly report their capabilities.

[lawadr]

That's Google's software decoder you've quoted. If I remember correctly, c2.android.avc.decoder is an alias for OMX.google.h264.decoder. If you keep looking you should find the hardware decoder. Could you post the report of that one please? It would be good to have it here for reference.

[MichaelRUSF]

Here's the full output for the Fire TV Stick 4k 2nd gen

[lawadr]

For AVC, those logs say it supports up to 5.2 High but doesn't support High 10 at all. For HEVC, it supports up to 5.1 Main 10.

The device page you linked to agrees on the level of HEVC support, but claims support for AVC High 10 at level 5.2. Which one is wrong I have no idea.

My suggestion would be to continue the special casing of AVC profile levels per device when support is different from what MediaCodec reports. In fact, this could be expanded to be on a per-profile basis rather than "global" across all AVC profiles.

[jellyfin-bot]

This issue has gone 120 days without comment. To avoid abandoned issues, it will be closed in 21 days if there are no new comments.

If you're the original submitter of this issue, please comment confirming if this issue still affects you in the latest release or master branch, or close the issue if it has been fixed. If you're another user also affected by this bug, please comment confirming so. Either action will remove the stale label.

This bot exists to prevent issues from becoming stale and forgotten. Jellyfin is always moving forward, and bugs are often fixed as side effects of other changes. We therefore ask that bug report authors remain vigilant about their issues to ensure they are closed if fixed, or re-confirmed - perhaps with fresh logs or reproduction examples - regularly. If you have any questions you can reach us on Matrix or Social Media.