Custom Device Profile(s)

[JPKribs]

Describe the feature you'd like Have Swiftfin default to a standard, universally functional Device Profile, but allow users to manually specify/override with their own profile for better compatibility.

Additional context Currently, Apple does a good job of keeping devices fairly standard but recently I have found there are some issues with codecs like AV1 between devices. Officially, Apple does not support AV1 hardware decoding on currently available Apple TVs. However, using players like Infuse, Software Decoding appears to work great for some models. 4K gen 1 and 2 stutter and playback is very poor but gen 3 playback works great with the software decoder. Additionally, many iPhones/iPads can software decode AV1 without issue, with the M4/A17 SOCs having hardware accelerated playback.

As a result of this, it's probably best for Swiftfin to default to transcoding AV1 content on the Server but allowing users to manually enable it. Roku has gone the route of allowing users to manually enable/disables things like HEVC/AV1 but it might make the most sense to just allow users to make their own profile. This way, so long as the enum for Video/Audio codecs are up to date, users can try whatever settings make the most sense.

Additionally, this lets users with older devices to downgrade their profiles to force transcoding for lower powered devices.

I have a PR where I have been messing around with this. I will like this issue on that Draft PR for feedback!

[LePips]

Since this is all about media compatibility, we can change this to encompass everything when it comes to how media is played, including combining the direct play setting and providing that very compatible profile as a preset.

Just as a design prototype, I've come up with the following (but only Profiles appears for Custom selection). Designs are not final as they are missing footer descriptions and design could be changed.

Compatibility

Profile - Empty

Profile - 2 profiles + swipe to delete

I didn't make a prototype for the profile creation screen but that shouldn't be too difficult and doesn't need to be fancy. Since there can be multiple profiles used I don't see why not to make that the functionality which also covers the single-profile case and allows people "who know what they're doing" to do what they want to do. We also need to have protections in case the user selects Replace and a profile isn't defined as that seems like unexpected behavior.

For the Most Compatible option, we can make the profile as described in the PR description. For combining with Direct Play, remove it from Experimental. All of this will require new types and such.

[JPKribs]

@LePips This looks awesome! Do you want me to finish out the PR with what I have now and then I can look towards this or should I go straight to trying towards this?

[LePips]

Let's do this right at the beginning alongside the cleanup.

[JPKribs]

I've been working on this and I think my biggest snag is storing this. If we're allowing more than one profile, it makes the most sense to store it all together. Since we're allowing changes to Audio, Video, and Container, I thought making a struct to store it made the most sense. Something like:

//
// Swiftfin is subject to the terms of the Mozilla Public
// License, v2.0. If a copy of the MPL was not distributed with this
// file, you can obtain one at https://mozilla.org/MPL/2.0/.
//
// Copyright (c) 2024 Jellyfin & Jellyfin Contributors
//

import Defaults
import Foundation
import JellyfinAPI

struct PlaybackDeviceProfile: Codable {
    var type: DlnaProfileType
    var audio: [AudioCodec]
    var video: [VideoCodec]
    var container: [MediaContainer]

    init(
        type: DlnaProfileType,
        audio: [AudioCodec] = [],
        video: [VideoCodec] = [],
        container: [MediaContainer] = []
    ) {
        self.type = type
        self.audio = audio
        self.video = video
        self.container = container
    }

    var directPlayProfile: DirectPlayProfile {
        switch type {
        case .video:
            return DirectPlayProfile(
                audioCodec: AudioCodec.unwrap(audio),
                container: MediaContainer.unwrap(container),
                type: type,
                videoCodec: VideoCodec.unwrap(video)
            )
        default:
            assertionFailure("Only Video is currently supported.")
            return DirectPlayProfile()
        }
    }

    var transcodingProfile: TranscodingProfile {
        switch type {
        case .video:
            return TranscodingProfile(
                audioCodec: AudioCodec.unwrap(audio),
                isBreakOnNonKeyFrames: true,
                container: MediaContainer.unwrap(container),
                context: .streaming,
                maxAudioChannels: "8",
                minSegments: 2,
                protocol: StreamType.hls.rawValue,
                type: .video,
                videoCodec: VideoCodec.unwrap(video)
            )
        default:
            assertionFailure("Only Video is currently supported.")
            return TranscodingProfile()
        }
    }
}

Then we can just call the var depending on whether this is a directPlay or transcodingProfile. The type is almost always going to be video but I thought I would include that so down the road we'd have something in place if we ever start looking at audio.

Does this make sense to store our profiles as [PlaybackDeviceProfile]? If so, how would I represent that in StoredValues?

[LePips]

Yes, combining them into single objects makes the most sense.

[JPKribs]

I've made some progress on this. This feature is now entirely functional but I have some weird bugs I still need to work out. For some reason, I started on tvOS so that's just what I have done. I expect iOS to be easier once I have the quirks figured out.

Here is my current state:

Playback Quality Menu (Complete)

Custom Device Profiles Menu (Done but with a Bug)

Custom Device Profiles Menu (Bug)

Focusing I lose my text...

Custom Device Profiles Editor

Issues:

1. ~Focusing the Device Profile the text disappears. I can make a custom focus but I think I just need to rework my buttons.~
2. ~Missing Navigation header for the Custom Device Profiles Editor. No idea what's going on there.~
3. Routing back from Custom Device Profiles Editor to Custom Device Profiles Menu jerks between the 2 screens a couple times. I think I need to redo the routing.
4. Updating a value on the Custom Device Profiles Editor for 'use as transcoding profile' updates the variable but the label that says On/Off doesn't update

I'm committing everything I have now. I'm hoping to knock out the issues above before I move to iOS.