Force maximum quality causes some 4k content to transcode

[jh-792]

Enabling the REPLEX_FORCE_MAXIMUM_QUALITY flag causes some 4k content to transcode. With the flag enabled, these files transcode no matter what my player settings ask for (even original quality).

If I disable this flag or bypass Replex, I can play this content at original quality without any issues when I set video quality to maximum in my Plex settings.

This mostly seems to happen with remuxes. The only difference I can see between these files and the 4k content that plays fine is the bitrate / filesize.

I see another user had a similar issue, but seemingly with all 4k content. Could it be something to do with Plex measuring available bandwidth or the capabilities of the device being used for playback if we are having similar issues but they are having the problem with more files?

I checked my Plex server logs and found some differences between the queries.

With the flag disabled: GET /video/:/transcode/universal/decision?hasMDE=1&path=%2Flibrary%2Fmetadata%2F14584&mediaIndex=1&partIndex=0&protocol=http&fastSeek=1&directPlay=1&directStream=1&subtitleSize=75&audioBoost=100&location=wan&addDebugOverlay=0&autoAdjustQuality=0&directStreamAudio=1&advancedSubtitles=text&mediaBufferSize=157286&session=#&offset=0&subtitles=none&copyts=1&Accept-Language=en-GB (12 live) #36000 GZIP

With the flag enabled: GET /video/:/transcode/universal/decision?partIndex=0&audioBoost=100&session=#&autoAdjustQuality=0&addDebugOverlay=0&subtitleSize=75&protocol=http&fastSeek=1&offset=0&location=wan&hasMDE=0&path=%2Flibrary%2Fmetadata%2F14584&subtitles=none&copyts=0&videoQuality=100&mediaBufferSize=157286&directStreamAudio=1&directStream=1&Accept-Language=en-GB&mediaIndex=1&directPlay=1&advancedSubtitles=text (6 live) #365cb GZIP

Replex sets hasMDE and copyts to 0, and adds the videoQuality flag setting it to 100. I assume that this issue is caused by one or more of those flags.

[lostb1t]

Do you use the latest version? as Replex does not set hasmde and copyts anymore

[lostb1t]

And I have to admit I'm not sure what videoQuality actually does. Don't think it does anything to be honest. My guess it's a legacy value for older devices

[jh-792]

Thanks, I will pull the latest version and test.

I did wonder whether maybe videoQuality isn't on a scale of 1-100.

[jh-792]

Unfortunately the problem persists. It looks like the directPlay flag is being set to 0 now, which may be causing issues.

Log files with maximum quality flag set (server attempts to transcode): GET /video/:/transcode/universal/decision?directPlay=0&advancedSubtitles=text&autoAdjustQuality=0&hasMDE=1&partIndex=0&location=wan&mediaBufferSize=157286&offset=431&videoQuality=100&session=#&path=%2Flibrary%2Fmetadata%2F1245959&directStreamAudio=1&addDebugOverlay=0&directStream=1&mediaIndex=1&subtitleSize=75&Accept-Language=en-GB&fastSeek=1&protocol=http&copyts=1&subtitles=auto&videoResolution=4096x2160&audioBoost=100 (13 live) #4ad241 GZIP [Req#4ad241/Transcode] MDE: Spider-Man: Into the Spider-Verse (2018): Direct Play is disabled [Req#4ad241/Transcode] MDE: Spider-Man: Into the Spider-Verse (2018): no direct play video profile exists for http/mkv/hevc [Req#4ad241/Transcode] MDE: Spider-Man: Into the Spider-Verse (2018): no direct play video profile exists for http/mkv/hevc/truehd [Req#4ad241/Transcode] MDE: Spider-Man: Into the Spider-Verse (2018): no direct play video profile exists for http/mkv/hevc/ac3 [Req#4f6bb1/Transcode] Streaming Resource: Reached Decision id=1245959 codes=(General=1001,Direct play not available; Conversion OK. Direct Play=3000,App cannot direct play this item. Direct play is disabled. Transcode=1001,Direct play not available; Conversion OK.) media=(id=1537126 part=(id=1620948 decision=transcode container=mkv protocol=http streams=(Video=(id=6267060 decision=copy width=3840 height=2160) Audio=(id=6267061 decision=transcode bitrate=1032 encoder=libopus channels=8 rate=48000))))

Log files without maximum quality flag set (direct play works): GET /video/:/transcode/universal/decision?hasMDE=1&path=%2Flibrary%2Fmetadata%2F1245959&mediaIndex=1&partIndex=0&protocol=http&fastSeek=1&directPlay=1&directStream=1&subtitleSize=75&audioBoost=100&location=wan&addDebugOverlay=0&autoAdjustQuality=0&directStreamAudio=1&advancedSubtitles=text&mediaBufferSize=157286&session=#&offset=431&subtitles=none&copyts=1&Accept-Language=en-GB (14 live) #4e0c2a GZIP [Req#4e0c2a/Transcode] Streaming Resource: Reached Decision id=1245959 codes=(MDE=1000,Direct play OK.) media=(id=1537126 part=(id=1620948 decision=direct play protocol=http streams=(Video=(id=6267060 decision= width=3840 height=2160) Audio=(id=6267061 decision= channels=0 rate=0))))

[lostb1t]

In the Replex request it says subtitles auto.

In your non Replex requests it says subtitles none? So it doesn't seem to be the same request. Replex doesn't touch the subtitles parameter

It also has directPlay set 0. This isn't touched by Replex either.

Make sure to disable subtitles when testing

[jh-792]

I've done some further testing. It looks like on the latest version, I can now play the high bitrate 4k content if I specifically request original quality. But Replex won't stop Plex from attempting to transcode if I ask for it. This did not work previously - there was nothing I could do to make it not transcode this content.

My Plex client appears to request subtitles none when I ask for the original quality, but requests subtitles auto if I request a transcode.

On lower bitrate content (where there wasn't an issue before) it will deliver the original quality file even if I ask for a transcode. So this still works.

Interestingly I see very similar log file entries for the lower bitrate 4k content that works fine with this flag even though it works as expected.

Requesting transcode (but it delivers original quality): GET /video/:/transcode/universal/decision?location=wan&advancedSubtitles=text&videoResolution=4096x2160&path=%2Flibrary%2Fmetadata%2F1468509&directStreamAudio=1&subtitleSize=75&copyts=1&directStream=1&offset=85&fastSeek=1&mediaBufferSize=157286&partIndex=0&addDebugOverlay=0&hasMDE=1&audioBoost=100&session=#&mediaIndex=0&subtitles=auto&Accept-Language=en-GB&videoQuality=100&directPlay=0&autoAdjustQuality=0&protocol=http (7 live) #50b4b6 GZIP [Req#50b4b6/Transcode] MDE: Elemental (2023): Direct Play is disabled [Req#50b4b6/Transcode] MDE: Elemental (2023): no direct play video profile exists for http/mkv/hevc [Req#50b4b6/Transcode] MDE: Elemental (2023): no direct play video profile exists for http/mkv/hevc/eac3 [Req#50b4b6/Transcode] Streaming Resource: Reached Decision id=1468509 codes=(General=1001,Direct play not available; Conversion OK. Direct Play=3000,App cannot direct play this item. Direct play is disabled. Transcode=1001,Direct play not available; Conversion OK.) media=(id=1836617 part=(id=1952825 decision=transcode container=mkv protocol=http streams=(Video=(id=7758483 decision=copy width=3840 height=2160) Audio=(id=7758485 decision=copy channels=6 rate=48000))))

Requesting original quality: GET /video/:/transcode/universal/decision?audioBoost=100&fastSeek=1&directPlay=1&directStreamAudio=1&copyts=1&path=%2Flibrary%2Fmetadata%2F1468509&offset=105&Accept-Language=en-GB&partIndex=0&addDebugOverlay=0&subtitles=none&videoResolution=4096x2160&mediaIndex=0&hasMDE=1&directStream=1&session=#&autoAdjustQuality=0&mediaBufferSize=157286&videoQuality=100&subtitleSize=75&advancedSubtitles=text&protocol=http&location=wan (8 live) #50c2fc GZIP) [Req#50c2fc/Transcode] Streaming Resource: Reached Decision id=1468509 codes=(MDE=1000,Direct play OK.) media=(id=1836617 part=(id=1952825 decision=direct play protocol=http streams=(Video=(id=7758483 decision= width=3840 height=2160) Audio=(id=7758485 decision= channels=0 rate=0))))

[lostb1t]

My Plex client appears to request subtitles none when I ask for the original quality, but requests subtitles auto if I request a transcode.

Is this the case for both with Replex on and off?

[jh-792]

Is this the case for both with Replex on and off?

Yes

[lostb1t]

And I still see subtitles requested. Subtitles can trigger a transcode so don't use subtitles when testing

[lostb1t]

The thing also to remember js that it doesn't prevent transcoding. It just removes the bitrate limit. So if a subtitle needs to be burned in or audio needs to be transcoded (or hdr to sdr) it will still transcode. Just not the bitrate. And a forced maximum will always direct stream not directPlay

Do you have a screenshot of tautulli if you use it?

[jh-792]

I can play it at original quality with the PGS subtitle track turned on and it doesn't attempt to transcode and burn the subs in. So I don't think it's to do with the subtitles.

I do have tautulli but can't get anything useful out of it - playback is being killed by killstream upstream of me as 4k transcoding is forbidden.

[lostb1t]

Looking at your log. The decision for audio and video is copy aka streaming. Not transcode so it's weird it's transcoding. But I bet if you disable subtitles it will not transcode.

[jh-792]

That would be surprising as I am able to play the original quality video with the subtitle track switched on without it triggering transcoding. Due to killstream kicking in I'm not able to make any changes to the subtitle settings before it stops playback.

Is it not possible to get Replex to replicate that query Plex player sends when it requests original quality?

[lostb1t]

The client asks for a transcode so disables directPlay.

The issue is that I don't know if the client can actually directPlay. Maybe it needs to transcode hdr to sdr or hevc to h264. Or burn in subtitles.

So it's only direct streaming letting plex handle the decisions. But I'm guessing because it's not a directplay plex decides to burn in the subtitles. But I'll have to do some testing to see if this is the case.

I could try to enable directPlay. But I'm unsure of the consequences like I mentioned

[jh-792]

Interesting. Is it possible to add a new flag to disable 4k video transcoding entirely? Would it be possible to do this while allowing audio transcode? I think it would be a very useful feature for many people who have servers that aren't beefy enough to support 4k transcoding at all.

So you'd have two flags. One that behaves as it currently does to minimize 4k transcoding, and one that forbids all 4k transcoding and will force direct play for such content (but allow audio transcoding)

[lostb1t]

Yeah that's what I'm thinking aswl. Have groups for the qualities so 4k and 1080p can be changed seperatly

[lostb1t]

@jh-792 one question: what client are you using?

[lostb1t]

k added REPLEX_FORCE_DIRECT_PLAY_FOR see readme for info on it.

[jh-792]

I am using Plex Desktop on Windows.

Thanks for the add! Unfortunately, the two new flags don't seem to be working for me, running the 0.17.9 docker image.

With REPLEX_FORCE_DIRECT_PLAY_FOR set to 4k, I get the following messages in the log when attempting to play a 4k movie without my Plex settings set to request maximum quality (so it automatically requests a 1080p 12Mbps transcode)

GET /video/:/transcode/universal/decision?hasMDE=1&path=%2Flibrary%2Fmetadata%2F1245959&mediaIndex=0&partIndex=0&protocol=http&fastSeek=1&directPlay=0&directStream=1&subtitleSize=75&audioBoost=100&location=wan&maxVideoBitrate=12000&addDebugOverlay=0&autoAdjustQuality=0&directStreamAudio=1&advancedSubtitles=text&mediaBufferSize=157286&session=#&offset=856&subtitles=auto&copyts=1&Accept-Language=en-GB (12 live) #666 GZIP [Req#666/Transcode] Streaming Resource: Calculated bandwidth of 33728kbps exceeds bandwidth limit. Changing decision parameters provided by client to fit bandwidth limit of 12000kbps

This applies to all 4k content (both high and low bitrate).

REPLEX_FORCE_MAXIMUM_QUALITY is behaving as it was before (triggering a transcode for high bitrate 4k content), but I'm assuming that's expected.

[lostb1t]

New release had a breaking bug. Should be good to go now

[eandrade92630]

I fixed this issue in my own code but I see you've updated a lot of things and are using different end points now. I'm curious as to what the differences are, it used to be decisions and now you're using session. Do the same parameters get passed to this end point? Haven't had the time to investigate myself. I briefly skimmed over your code but I don't know rust and I'm having a hard time understanding some of it (mostly cause there's a lot).

[lostb1t]

it is still using decision. The sessions endpoint is only for redirecting streams. Not for transcoding.

[lostb1t]

relevant code for direct play is here: https://github.com/lostb1t/replex/blob/main/src/routes.rs#L650

[eandrade92630]

So the routing list should include all of /video/:/transcode/universal? I also noticed /hubs and /replex were removed, are these not used anymore? I haven't gotten the chance to try the latest version but in case it hasn't been fixed yet I figured I'd share what I found here. Note: some of this may negate the need for my script anymore as it looks like you've added some recent features that could be of high value to me. In studying the parameters sent to the decision end point I've found this, directStream is an indicator of not just if the client is capable of direct streaming, but also whether their Remote Quality Streaming settings are set to always transcode to a lesser bitrate or if it's set to max quality. When a bitrate is chosen directPlay is set to 1, when original quality is chosen directPlay is set to 0. videoBitrate and maxVideoBitrate are the bitrates the client is attempting to stream at (eg 1080p high = 20,000). The decision endpoint uses this information to determine whether to transcode or not. Using this I created logic so that if a user has not set their streaming settings to stream at original quality they will always be forced to stream at original quality ((directPlay OR directStream = 1) AND (videoBitrate OR maxVideoBitrate =< 20,000)) = force Max quality, however, I didn't want to punish my friends who are literate enough to flip the switch, so I have logic that says if ((directPlay OR directStream = 0) AND (videoBitrate OR maxVideoBitrate =< 20,000) = transcoding allowed, pass through original parameters. Previously 4K would transcode regardless of if the parameters with original and I still can't wrap my head around why, my solution? If bitrate = 200,000 (4K/2160) redirect the decision endpoint to 403 so a decision to transcode can't be made. I haven't done through testing so I can't tell how this will affect other users and different environments. Obviously some of the new features you've introduced negates my need for some of this logic but I thought I'd share with you my research.

[eandrade92630]

Sorry, it seems I didn't read the section for reverse proxies correctly. What you recommend to do now is route everything through replex EXCEPT those two paths?

[lostb1t]

yes that's what I suggest. Because routes change a lot during development. But if you want you can just route everything through replex. Those 2 paths in the readme are only for streaming which allows for a small performance boost if not routed through replex.

[eandrade92630]

Second note, just in case anyone else wanted to read. I first thought these parameters were requests to the decision endpoint for how the client wants to stream. They're not, they're device capabilities and the decision end point uses these to decide whether or not to transcode based on differences in capabilities and media requested. directPlay and directStream should not have to be modified (I've seen this suggested elsewhere), they are simply indicators of whether the client can or is allowed to direct stream or not. videoBitrate and maxVideoBitrate are the max bitrate the device can handle, I believe they are different but similar but I've always seen these two values the same so I have no idea what would make them different, but other than a client supporting only up to a certain bitrate the only other reason this would change is because the client specifically requests a certain bitrate. Anyhow, the only thing that has to be changed here is the bitrate, we're tricking the server to serve us the full quality video by telling it the client supports (or requests) 200,000, even if the client requests a lower bitrate such as 3,000 for example.

[lostb1t]

maxVideoBitrate has been used in the pest by clients to define the bitrate set in the settings. While videoBitrate would be the bitrate set during playing a video (so a single session) unfortunately maxVideoBitrate isn't used anymore in modern clients, only videoBitrate