search

jellyfin / jellyfin-roku

The Official Roku Client for Jellyfin
https://jellyfin.org
GNU General Public License v2.0
450 stars 141 forks source link

Scrambled OPUS Audio #267

Closed illeatmyhat closed 3 years ago

illeatmyhat commented 4 years ago

When playing an .MKV file with a 720p HEVC video stream, and a "Jpn OPUS 5.1 Default" audio stream, the audio is heavily scrambled. Mostly, only loud static noise can be heard. The natural sound from the audio stream can be barely heard, though also scrambled.

Works fine on the web version of Jellyfin. Probably a transcoding error. May be related to #231

Artiume commented 4 years ago

Seems like 5.1 audio isn't properly downmixing to Stereo

jcdick1 commented 3 years ago

Seems like 5.1 audio isn't properly downmixing to Stereo

It's both multi-channel and two-channel audio. I have several items in my library that have two audio tracks encoded with Opus, 6 channel and 2 channel. They are both heavily scrambled on my Roku Ultra.

Artiume commented 3 years ago

ok. I just tested 2ch Opus on my system and it plays fine. I wonder what's up with your system

neilsb commented 3 years ago

I've uploaded a couple of sample OPUS files here (about 8Mb each, one 6 channel and one stereo) that I've validated both work on my Roku with App v 1.4.5.

Could you try downloading these files and testing them on your Roku just to confirm if both, one or none sound garbled please?

jcdick1 commented 3 years ago

I downloaded the samples, and the stereo file plays fine, but the 5.1 is junky audio. But mine continue to be junk for both. Maybe it has something to do with how I used Handbrake to encode the Opus stereo track. I set it to "Dolby Pro Logic II" instead of regular stereo. I'll try one encoded as strictly stereo.

neilsb commented 3 years ago

OK thanks for that. Something for me to have a look at.

Do you have an 5.1 sound system connected, or is the actual output stereo?

jcdick1 commented 3 years ago

I have an Ultra, with HDMI audio set to (DD+,DTS) to reduce audio transcoding, and connected to an LG TV, so just stereo.

jcdick1 commented 3 years ago

I did have a shower thought last night about it being something related to not downmixing 5.1 to stereo properly, and this isn't specific to this issue, but ...

Since what audio can be direct played is somewhat determined by what audio capabilities are reported by the device the Roku is connected to, it would be kind of cool if the Roku client could look at the audio device, look at the available audio streams, and if the content contains both a surround and a stereo stream, automatically use the one that is most appropriate, with the user having the ability to override it. So if the Roku is connected to something that can only deal with stereo, the client automatically uses the stereo stream.

Artiume commented 3 years ago

https://developer.roku.com/en-gb/docs/references/brightscript/interfaces/ifdeviceinfo.md#candecodevideovideo_format-as-object-as-object

https://github.com/jellyfin/jellyfin-roku/blob/bab495271f08223fd739b9220eb148f9945eb6ff/source/utils/deviceCapabilities.brs#L185

This is where we query the roku device asking if it can support opus. It should propeely downmix 5.1 to stereo on the roku itself. I can play 5.1 audio (i havent tested the 5.1 opus sample provided yet) on my stereo only tv and it comes out fine. Have you isolated this issue to the jellyfin app specifically?

jcdick1 commented 3 years ago

I just went and checked with my Plex client, and it also sounds horrible using the Opus 5.1 stream.

Artiume commented 3 years ago

I set it to "Dolby Pro Logic II" instead of regular stereo.

This might be the issue but I'm not sure. I'd have to dig into the encoding requirements of the opus spec

jcdick1 commented 3 years ago

I can play 5.1 audio (i havent tested the 5.1 opus sample provided yet) on my stereo only tv and it comes out fine.

Did you encode your Opus 5.1 in Handbrake? It may be that OP and I used it, and it has something to do with Handbrake's settings for FFMpeg for encoding Opus.

Artiume commented 3 years ago

i want to say it was 5.1 aac. it hadnt been for this issue specifically. once I'm home i can test the sample that @neilsb provided. I dont really mess with Handbrake, I use FFmpeg directly haha

jcdick1 commented 3 years ago

i want to say it was 5.1 aac. it hadnt been for this issue specifically.

Oh, yeah. I can play 5.1 in other codecs just fine, as well. It's just Opus that's an issue. I'm going to say its either how FFmpeg encodes, Handbrake encodes, or the Roku handles it, but probably not inherent to the JF client.

Artiume commented 3 years ago

can you pull the exact ffmpeg command that handbrake uses or provide a sample? I'd like to check it out. if not, the output of ffmpeg -i file might be enough

jcdick1 commented 3 years ago

I pulled this out of Kodi's video sample library, and crunched it through Handbrake real quick. I transcoded it from VC-1 with TrueHD 5.1 to VP9 with 3 audio tracks, Opus 5.1 at 640k, Opus using the Dolby Pro Logic II setting at 160k, and then just straight stereo, also at 160k. All audio streams are a mess on my Roku/LG setup.

Opus_in_three_streams.zip

Artiume commented 3 years ago

Is it transcoding? or is it Direct Playing?

jcdick1 commented 3 years ago

The server dashboard shows it as Direct Playing for all three audio streams.

neilsb commented 3 years ago

So a little more testing shows that even when a stereo output source is connected, the Roku is telling Jellyfin it supports 6 channel Open (and AAC amongst others). So the interference is almost certainly to do with the Roku device downmixing to Stereo. I don't have 5.1 decoder to test if it handles 5.1 OPUS correctly if it's not performing downmixing.

The same behaviour is seen with aac as well, but Roku seems to downmix that fine.

The suggestion about the Roku looking at the detected (or manually set) output settings and defaulting to the best stream is an interesting idea, and makes sense. Might be worth raising a separate issue for this as it would be a nice to have, whereas sorting Audio not working is more of a priority. Wouldn't want the idea getting lost when this issue is closed.

Artiume commented 3 years ago

ow. that hurt. I got the same results. All 3 audio clips from your sample were bad. Neils' 2ch was good and the 6ch failed to properly downmix but it wasn't nearly as bad as your samples. You could at least hear Neils' audio whereas with yours it sounded like a complete encoding failure.

Artiume commented 3 years ago
Good OPUS 2 channel > [STREAM] > index=1 > codec_name=opus > codec_long_name=Opus (Opus Interactive Audio Codec) > profile=unknown > codec_type=audio > codec_time_base=1/48000 > codec_tag_string=[0][0][0][0] > codec_tag=0x0000 > sample_fmt=fltp > sample_rate=48000 > channels=2 > channel_layout=stereo > bits_per_sample=0 > id=N/A > r_frame_rate=0/0 > avg_frame_rate=0/0 > time_base=1/1000 > start_pts=-7 > start_time=-0.007000 > duration_ts=N/A > duration=N/A > bit_rate=N/A > max_bit_rate=N/A > bits_per_raw_sample=N/A > nb_frames=N/A > nb_read_frames=N/A > nb_read_packets=N/A > DISPOSITION:default=1 > DISPOSITION:dub=0 > DISPOSITION:original=0 > DISPOSITION:comment=0 > DISPOSITION:lyrics=0 > DISPOSITION:karaoke=0 > DISPOSITION:forced=0 > DISPOSITION:hearing_impaired=0 > DISPOSITION:visual_impaired=0 > DISPOSITION:clean_effects=0 > DISPOSITION:attached_pic=0 > DISPOSITION:timed_thumbnails=0 > TAG:title=Stereo > TAG:DURATION=00:00:16.241000000 > [/STREAM]
Good Opus 5.1, but not properly downmixed > [STREAM] > index=1 > codec_name=opus > codec_long_name=Opus (Opus Interactive Audio Codec) > profile=unknown > codec_type=audio > codec_time_base=1/48000 > codec_tag_string=[0][0][0][0] > codec_tag=0x0000 > sample_fmt=fltp > sample_rate=48000 > channels=6 > channel_layout=5.1 > bits_per_sample=0 > id=N/A > r_frame_rate=0/0 > avg_frame_rate=0/0 > time_base=1/1000 > start_pts=-7 > start_time=-0.007000 > duration_ts=N/A > duration=N/A > bit_rate=N/A > max_bit_rate=N/A > bits_per_raw_sample=N/A > nb_frames=N/A > nb_read_frames=N/A > nb_read_packets=N/A > DISPOSITION:default=1 > DISPOSITION:dub=0 > DISPOSITION:original=0 > DISPOSITION:comment=0 > DISPOSITION:lyrics=0 > DISPOSITION:karaoke=0 > DISPOSITION:forced=0 > DISPOSITION:hearing_impaired=0 > DISPOSITION:visual_impaired=0 > DISPOSITION:clean_effects=0 > DISPOSITION:attached_pic=0 > DISPOSITION:timed_thumbnails=0 > TAG:title=Surround > TAG:DURATION=00:00:16.241000000 > [/STREAM]
Bad Opus 5.1ch > [STREAM] > index=1 > codec_name=opus > codec_long_name=Opus (Opus Interactive Audio Codec) > profile=unknown > codec_type=audio > codec_time_base=1/48000 > codec_tag_string=[0][0][0][0] > codec_tag=0x0000 > sample_fmt=fltp > sample_rate=48000 > channels=6 > channel_layout=5.1 > bits_per_sample=0 > id=N/A > r_frame_rate=0/0 > avg_frame_rate=0/0 > time_base=1/1000 > start_pts=-7 > start_time=-0.007000 > duration_ts=N/A > duration=N/A > bit_rate=N/A > max_bit_rate=N/A > bits_per_raw_sample=N/A > nb_frames=N/A > nb_read_frames=N/A > nb_read_packets=N/A > DISPOSITION:default=1 > DISPOSITION:dub=0 > DISPOSITION:original=0 > DISPOSITION:comment=0 > DISPOSITION:lyrics=0 > DISPOSITION:karaoke=0 > DISPOSITION:forced=0 > DISPOSITION:hearing_impaired=0 > DISPOSITION:visual_impaired=0 > DISPOSITION:clean_effects=0 > DISPOSITION:attached_pic=0 > DISPOSITION:timed_thumbnails=0 > TAG:language=eng > TAG:title=Opus 5.1 > TAG:DURATION=00:00:59.981000000 > [/STREAM]
Opus 2ch With Dolby Pro Logic II mix > [STREAM] > index=2 > codec_name=opus > codec_long_name=Opus (Opus Interactive Audio Codec) > profile=unknown > codec_type=audio > codec_time_base=1/48000 > codec_tag_string=[0][0][0][0] > codec_tag=0x0000 > sample_fmt=fltp > sample_rate=48000 > channels=2 > channel_layout=stereo > bits_per_sample=0 > id=N/A > r_frame_rate=0/0 > avg_frame_rate=0/0 > time_base=1/1000 > start_pts=-7 > start_time=-0.007000 > duration_ts=N/A > duration=N/A > bit_rate=N/A > max_bit_rate=N/A > bits_per_raw_sample=N/A > nb_frames=N/A > nb_read_frames=N/A > nb_read_packets=N/A > DISPOSITION:default=0 > DISPOSITION:dub=0 > DISPOSITION:original=0 > DISPOSITION:comment=0 > DISPOSITION:lyrics=0 > DISPOSITION:karaoke=0 > DISPOSITION:forced=0 > DISPOSITION:hearing_impaired=0 > DISPOSITION:visual_impaired=0 > DISPOSITION:clean_effects=0 > DISPOSITION:attached_pic=0 > DISPOSITION:timed_thumbnails=0 > TAG:language=eng > TAG:title=With Dolby Pro Logic II mix > TAG:DURATION=00:00:59.981000000 > [/STREAM]
Opus 2ch bad > [STREAM] > index=3 > codec_name=opus > codec_long_name=Opus (Opus Interactive Audio Codec) > profile=unknown > codec_type=audio > codec_time_base=1/48000 > codec_tag_string=[0][0][0][0] > codec_tag=0x0000 > sample_fmt=fltp > sample_rate=48000 > channels=2 > channel_layout=stereo > bits_per_sample=0 > id=N/A > r_frame_rate=0/0 > avg_frame_rate=0/0 > time_base=1/1000 > start_pts=-7 > start_time=-0.007000 > duration_ts=N/A > duration=N/A > bit_rate=N/A > max_bit_rate=N/A > bits_per_raw_sample=N/A > nb_frames=N/A > nb_read_frames=N/A > nb_read_packets=N/A > DISPOSITION:default=0 > DISPOSITION:dub=0 > DISPOSITION:original=0 > DISPOSITION:comment=0 > DISPOSITION:lyrics=0 > DISPOSITION:karaoke=0 > DISPOSITION:forced=0 > DISPOSITION:hearing_impaired=0 > DISPOSITION:visual_impaired=0 > DISPOSITION:clean_effects=0 > DISPOSITION:attached_pic=0 > DISPOSITION:timed_thumbnails=0 > TAG:language=eng > TAG:title=Stereo > TAG:DURATION=00:00:59.981000000 > [/STREAM]

Only difference I see is TAG:language=eng which I doubt would cause the issue. I'll look to see how else I can analyze the differences.

I found this open ticket for opus in mkv.

https://trac.ffmpeg.org/ticket/4178

jd1123 commented 3 years ago

I am having problems with the Roku app (not the web client however) and OPUS as well. I get no audio from two TVs with Rokus on them when the audio format is OPUS. The video stream is transcoded, but the audio stream is copied (from logs). If I change the audio stream to something else (FLAC for instance) with ffmpeg ahead of time, it works just fine.

Artiume commented 3 years ago

if you test exo in Android, you will probably get the same results. i think the issue is upstream of jellyfin

jcdick1 commented 3 years ago

I just tried a new encode of VP9/Opus 5.1 on my Roku (OS 10.0.0 b4209) with JF 1.4.b8 and while it plays fine in the JMP on the desktop and the Android client (2.3.3), it is still just noise on the Roku, the same as a year ago. Just to provide an update.