Audio stuttering when transcode playing ogg vorbis on iOS

felix920506 commented 5 months ago

Description When turning on the "Transcode" option in settings and playing Ogg Vorbis audio, the audio stutters. When transcoding from other source formats this doesn't happen

Media Information

last
Container: ogg
Path: /Users/user/Music/test_music_lib/Arcaea All Songs/last.ogg
Size: 4 MB

Audio
Title: Last - VORBIS - Stereo
Codec: VORBIS
AVC: No
Layout: stereo
Channels: 2 ch
Bitrate: 192 kbps
Sample rate: 44100 Hz
Video range: Unknown
Video range type: Unknown
Default: No
Forced: No
External: No

Image
Codec: MJPEG
AVC: No
Profile: Baseline
Resolution: 768x768
Bit depth: 8 bit
Video range: Unknown
Video range type: Unknown
Color space: bt470bg
Pixel format: yuvj444p
Ref frames: 1

Chaphasilor commented 4 months ago

Hi, could you take a look at the transcoding logs of your Jellyfin server? I know that transcoding to Vorbis on iOS is problematic, and maybe the server isn't transcoding at all...

I noticed the bitrate of the file is 192 kbps, what do you have your transcoding bitrate set to? There was an issue (https://github.com/jmshrv/finamp/issues/269) where lower-bitrate tracks weren't being transcoded, but that should be fixed already...

felix920506 commented 4 months ago

@Chaphasilor I'm not transcoding to Vorbis. I'm transcoding from Vorbis as Vorbis won't direct play.

I have transcoding bitrate set to 320kbps.

Here is a sample ffmpeg log

{"Protocol":0,"Id":"feb56d9b05dab1faabb53c433f74169a","Path":"/Users/user/Music/test_music_lib/Arcaea All Songs/last.ogg","EncoderPath":null,"EncoderProtocol":null,"Type":0,"Container":"ogg","Size":3944557,"Name":"last","IsRemote":false,"ETag":"509a90d84c357b021e3cfd83ae1efa37","RunTimeTicks":1474285710,"ReadAtNativeFramerate":false,"IgnoreDts":false,"IgnoreIndex":false,"GenPtsInput":false,"SupportsTranscoding":true,"SupportsDirectStream":true,"SupportsDirectPlay":true,"IsInfiniteStream":false,"RequiresOpening":false,"OpenToken":null,"RequiresClosing":false,"LiveStreamId":null,"BufferMs":null,"RequiresLooping":false,"SupportsProbing":true,"VideoType":null,"IsoType":null,"Video3DFormat":null,"MediaStreams":[{"Codec":"vorbis","CodecTag":null,"Language":null,"ColorRange":null,"ColorSpace":null,"ColorTransfer":null,"ColorPrimaries":null,"DvVersionMajor":null,"DvVersionMinor":null,"DvProfile":null,"DvLevel":null,"RpuPresentFlag":null,"ElPresentFlag":null,"BlPresentFlag":null,"DvBlSignalCompatibilityId":null,"Comment":null,"TimeBase":"1/44100","CodecTimeBase":null,"Title":"Last","VideoRange":0,"VideoRangeType":0,"VideoDoViTitle":null,"AudioSpatialFormat":0,"LocalizedUndefined":null,"LocalizedDefault":null,"LocalizedForced":null,"LocalizedExternal":null,"LocalizedHearingImpaired":null,"DisplayTitle":"Last - VORBIS - Stereo","NalLengthSize":null,"IsInterlaced":false,"IsAVC":false,"ChannelLayout":"stereo","BitRate":192000,"BitDepth":null,"RefFrames":null,"PacketLength":null,"Channels":2,"SampleRate":44100,"IsDefault":false,"IsForced":false,"IsHearingImpaired":false,"Height":null,"Width":null,"AverageFrameRate":null,"RealFrameRate":null,"Profile":null,"Type":0,"AspectRatio":null,"Index":0,"Score":null,"IsExternal":false,"DeliveryMethod":null,"DeliveryUrl":null,"IsExternalUrl":null,"IsTextSubtitleStream":false,"SupportsExternalStream":false,"Path":null,"PixelFormat":null,"Level":0,"IsAnamorphic":null},{"Codec":"mjpeg","CodecTag":null,"Language":null,"ColorRange":null,"ColorSpace":"bt470bg","ColorTransfer":null,"ColorPrimaries":null,"DvVersionMajor":null,"DvVersionMinor":null,"DvProfile":null,"DvLevel":null,"RpuPresentFlag":null,"ElPresentFlag":null,"BlPresentFlag":null,"DvBlSignalCompatibilityId":null,"Comment":"Cover (front)","TimeBase":"1/90000","CodecTimeBase":null,"Title":null,"VideoRange":0,"VideoRangeType":0,"VideoDoViTitle":null,"AudioSpatialFormat":0,"LocalizedUndefined":null,"LocalizedDefault":null,"LocalizedForced":null,"LocalizedExternal":null,"LocalizedHearingImpaired":null,"DisplayTitle":null,"NalLengthSize":null,"IsInterlaced":false,"IsAVC":false,"ChannelLayout":null,"BitRate":null,"BitDepth":8,"RefFrames":1,"PacketLength":null,"Channels":null,"SampleRate":null,"IsDefault":false,"IsForced":false,"IsHearingImpaired":false,"Height":768,"Width":768,"AverageFrameRate":null,"RealFrameRate":90000,"Profile":"Baseline","Type":3,"AspectRatio":"1:1","Index":1,"Score":null,"IsExternal":false,"DeliveryMethod":null,"DeliveryUrl":null,"IsExternalUrl":null,"IsTextSubtitleStream":false,"SupportsExternalStream":false,"Path":null,"PixelFormat":"yuvj444p","Level":-99,"IsAnamorphic":false}],"MediaAttachments":[],"Formats":[],"Bitrate":214045,"Timestamp":null,"RequiredHttpHeaders":{},"TranscodingUrl":null,"TranscodingSubProtocol":null,"TranscodingContainer":null,"AnalyzeDurationMs":null,"DefaultAudioStreamIndex":null,"DefaultSubtitleStreamIndex":null}

ffmpeg -analyzeduration 200M -probesize 1G  -i file:"/Users/user/Music/test_music_lib/Arcaea All Songs/last.ogg" -map_metadata -1 -map_chapters -1 -threads 0   -vn -acodec aac_at -ab 256000 -ac 2 -copyts -avoid_negative_ts disabled -max_muxing_queue_size 2048 -f hls -max_delay 5000000 -hls_time 3 -hls_segment_type mpegts -start_number 0 -hls_segment_filename "/Users/user/Library/Application Support/jellyfin/cache/transcodes/ae278c66fb6a8a9caca834951109e78c%d.ts" -hls_playlist_type vod -hls_list_size 0 -y "/Users/user/Library/Application Support/jellyfin/cache/transcodes/ae278c66fb6a8a9caca834951109e78c.m3u8"

ffmpeg version 6.1.1 Copyright (c) 2000-2023 the FFmpeg developers
  built with Apple clang version 15.0.0 (clang-1500.1.0.2.5)
  configuration: --prefix=/opt/homebrew/Cellar/ffmpeg/6.1.1_3 --enable-shared --enable-pthreads --enable-version3 --cc=clang --host-cflags= --host-ldflags='-Wl,-ld_classic' --enable-ffplay --enable-gnutls --enable-gpl --enable-libaom --enable-libaribb24 --enable-libbluray --enable-libdav1d --enable-libharfbuzz --enable-libjxl --enable-libmp3lame --enable-libopus --enable-librav1e --enable-librist --enable-librubberband --enable-libsnappy --enable-libsrt --enable-libssh --enable-libsvtav1 --enable-libtesseract --enable-libtheora --enable-libvidstab --enable-libvmaf --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libxvid --enable-lzma --enable-libfontconfig --enable-libfreetype --enable-frei0r --enable-libass --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopenvino --enable-libspeex --enable-libsoxr --enable-libzmq --enable-libzimg --disable-libjack --disable-indev=jack --enable-videotoolbox --enable-audiotoolbox --enable-neon
  libavutil      58. 29.100 / 58. 29.100
  libavcodec     60. 31.102 / 60. 31.102
  libavformat    60. 16.100 / 60. 16.100
  libavdevice    60.  3.100 / 60.  3.100
  libavfilter     9. 12.100 /  9. 12.100
  libswscale      7.  5.100 /  7.  5.100
  libswresample   4. 12.100 /  4. 12.100
  libpostproc    57.  3.100 / 57.  3.100
[ogg @ 0x14b606110] 4602 bytes of comment header remain
Input #0, ogg, from 'file:/Users/user/Music/test_music_lib/Arcaea All Songs/last.ogg':
  Duration: 00:02:27.43, start: 0.000000, bitrate: 214 kb/s
  Stream #0:0: Audio: vorbis, 44100 Hz, stereo, fltp, 192 kb/s
    Metadata:
      ENCODER         : REAPER
      TITLE           : Last
      ARTIST          : onoken
  Stream #0:1: Video: mjpeg (Baseline), yuvj444p(pc, bt470bg/unknown/unknown), 768x768 [SAR 1:1 DAR 1:1], 90k tbr, 90k tbn (attached pic)
    Metadata:
      comment         : Cover (front)
Stream mapping:
  Stream #0:0 -> #0:0 (vorbis (native) -> aac (aac_at))
Press [q] to stop, [?] for help
[mpegts @ 0x14b6147d0] frame size not set
Output #0, hls, to '/Users/user/Library/Application Support/jellyfin/cache/transcodes/ae278c66fb6a8a9caca834951109e78c.m3u8':
  Metadata:
    encoder         : Lavf60.16.100
  Stream #0:0: Audio: aac, 44100 Hz, stereo, s16, 256 kb/s
    Metadata:
      encoder         : Lavc60.31.102 aac_at
size=       0kB time=N/A bitrate=N/A speed=N/A    
[hls @ 0x14b607af0] Opening '/Users/user/Library/Application Support/jellyfin/cache/transcodes/ae278c66fb6a8a9caca834951109e78c0.ts' for writing
[hls @ 0x14b607af0] Opening '/Users/user/Library/Application Support/jellyfin/cache/transcodes/ae278c66fb6a8a9caca834951109e78c1.ts' for writing
[hls @ 0x14b607af0] Opening '/Users/user/Library/Application Support/jellyfin/cache/transcodes/ae278c66fb6a8a9caca834951109e78c2.ts' for writing
[hls @ 0x14b607af0] Opening '/Users/user/Library/Application Support/jellyfin/cache/transcodes/ae278c66fb6a8a9caca834951109e78c3.ts' for writing
[hls @ 0x14b607af0] Opening '/Users/user/Library/Application Support/jellyfin/cache/transcodes/ae278c66fb6a8a9caca834951109e78c4.ts' for writing
[hls @ 0x14b607af0] Opening '/Users/user/Library/Application Support/jellyfin/cache/transcodes/ae278c66fb6a8a9caca834951109e78c5.ts' for writing
[hls @ 0x14b607af0] Opening '/Users/user/Library/Application Support/jellyfin/cache/transcodes/ae278c66fb6a8a9caca834951109e78c6.ts' for writing
[hls @ 0x14b607af0] Opening '/Users/user/Library/Application Support/jellyfin/cache/transcodes/ae278c66fb6a8a9caca834951109e78c7.ts' for writing
[hls @ 0x14b607af0] Opening '/Users/user/Library/Application Support/jellyfin/cache/transcodes/ae278c66fb6a8a9caca834951109e78c8.ts' for writing
[hls @ 0x14b607af0] Opening '/Users/user/Library/Application Support/jellyfin/cache/transcodes/ae278c66fb6a8a9caca834951109e78c9.ts' for writing
[hls @ 0x14b607af0] Opening '/Users/user/Library/Application Support/jellyfin/cache/transcodes/ae278c66fb6a8a9caca834951109e78c10.ts' for writing
[hls @ 0x14b607af0] Opening '/Users/user/Library/Application Support/jellyfin/cache/transcodes/ae278c66fb6a8a9caca834951109e78c11.ts' for writing
[hls @ 0x14b607af0] Opening '/Users/user/Library/Application Support/jellyfin/cache/transcodes/ae278c66fb6a8a9caca834951109e78c12.ts' for writing
[hls @ 0x14b607af0] Opening '/Users/user/Library/Application Support/jellyfin/cache/transcodes/ae278c66fb6a8a9caca834951109e78c13.ts' for writing
[hls @ 0x14b607af0] Opening '/Users/user/Library/Application Support/jellyfin/cache/transcodes/ae278c66fb6a8a9caca834951109e78c14.ts' for writing
size=N/A time=00:00:00.00 bitrate=N/A speed=   0x    
[hls @ 0x14b607af0] Opening '/Users/user/Library/Application Support/jellyfin/cache/transcodes/ae278c66fb6a8a9caca834951109e78c15.ts' for writing
[hls @ 0x14b607af0] Opening '/Users/user/Library/Application Support/jellyfin/cache/transcodes/ae278c66fb6a8a9caca834951109e78c16.ts' for writing
[hls @ 0x14b607af0] Opening '/Users/user/Library/Application Support/jellyfin/cache/transcodes/ae278c66fb6a8a9caca834951109e78c17.ts' for writing
[hls @ 0x14b607af0] Opening '/Users/user/Library/Application Support/jellyfin/cache/transcodes/ae278c66fb6a8a9caca834951109e78c18.ts' for writing
[hls @ 0x14b607af0] Opening '/Users/user/Library/Application Support/jellyfin/cache/transcodes/ae278c66fb6a8a9caca834951109e78c19.ts' for writing
[hls @ 0x14b607af0] Opening '/Users/user/Library/Application Support/jellyfin/cache/transcodes/ae278c66fb6a8a9caca834951109e78c20.ts' for writing
[hls @ 0x14b607af0] Opening '/Users/user/Library/Application Support/jellyfin/cache/transcodes/ae278c66fb6a8a9caca834951109e78c21.ts' for writing
[hls @ 0x14b607af0] Opening '/Users/user/Library/Application Support/jellyfin/cache/transcodes/ae278c66fb6a8a9caca834951109e78c22.ts' for writing
[hls @ 0x14b607af0] Opening '/Users/user/Library/Application Support/jellyfin/cache/transcodes/ae278c66fb6a8a9caca834951109e78c23.ts' for writing
[hls @ 0x14b607af0] Opening '/Users/user/Library/Application Support/jellyfin/cache/transcodes/ae278c66fb6a8a9caca834951109e78c24.ts' for writing
[hls @ 0x14b607af0] Opening '/Users/user/Library/Application Support/jellyfin/cache/transcodes/ae278c66fb6a8a9caca834951109e78c25.ts' for writing
[hls @ 0x14b607af0] Opening '/Users/user/Library/Application Support/jellyfin/cache/transcodes/ae278c66fb6a8a9caca834951109e78c26.ts' for writing
[hls @ 0x14b607af0] Opening '/Users/user/Library/Application Support/jellyfin/cache/transcodes/ae278c66fb6a8a9caca834951109e78c27.ts' for writing
[hls @ 0x14b607af0] Opening '/Users/user/Library/Application Support/jellyfin/cache/transcodes/ae278c66fb6a8a9caca834951109e78c28.ts' for writing
[hls @ 0x14b607af0] Opening '/Users/user/Library/Application Support/jellyfin/cache/transcodes/ae278c66fb6a8a9caca834951109e78c29.ts' for writing
[hls @ 0x14b607af0] Opening '/Users/user/Library/Application Support/jellyfin/cache/transcodes/ae278c66fb6a8a9caca834951109e78c30.ts' for writing
size=N/A time=00:00:49.13 bitrate=N/A speed=49.1x    
[hls @ 0x14b607af0] Opening '/Users/user/Library/Application Support/jellyfin/cache/transcodes/ae278c66fb6a8a9caca834951109e78c31.ts' for writing
[hls @ 0x14b607af0] Opening '/Users/user/Library/Application Support/jellyfin/cache/transcodes/ae278c66fb6a8a9caca834951109e78c32.ts' for writing
[hls @ 0x14b607af0] Opening '/Users/user/Library/Application Support/jellyfin/cache/transcodes/ae278c66fb6a8a9caca834951109e78c33.ts' for writing
[hls @ 0x14b607af0] Opening '/Users/user/Library/Application Support/jellyfin/cache/transcodes/ae278c66fb6a8a9caca834951109e78c34.ts' for writing
[hls @ 0x14b607af0] Opening '/Users/user/Library/Application Support/jellyfin/cache/transcodes/ae278c66fb6a8a9caca834951109e78c35.ts' for writing
[hls @ 0x14b607af0] Opening '/Users/user/Library/Application Support/jellyfin/cache/transcodes/ae278c66fb6a8a9caca834951109e78c36.ts' for writing
[hls @ 0x14b607af0] Opening '/Users/user/Library/Application Support/jellyfin/cache/transcodes/ae278c66fb6a8a9caca834951109e78c37.ts' for writing
[hls @ 0x14b607af0] Opening '/Users/user/Library/Application Support/jellyfin/cache/transcodes/ae278c66fb6a8a9caca834951109e78c38.ts' for writing
[hls @ 0x14b607af0] Opening '/Users/user/Library/Application Support/jellyfin/cache/transcodes/ae278c66fb6a8a9caca834951109e78c39.ts' for writing
[hls @ 0x14b607af0] Opening '/Users/user/Library/Application Support/jellyfin/cache/transcodes/ae278c66fb6a8a9caca834951109e78c40.ts' for writing
[hls @ 0x14b607af0] Opening '/Users/user/Library/Application Support/jellyfin/cache/transcodes/ae278c66fb6a8a9caca834951109e78c41.ts' for writing
[hls @ 0x14b607af0] Opening '/Users/user/Library/Application Support/jellyfin/cache/transcodes/ae278c66fb6a8a9caca834951109e78c42.ts' for writing
[hls @ 0x14b607af0] Opening '/Users/user/Library/Application Support/jellyfin/cache/transcodes/ae278c66fb6a8a9caca834951109e78c43.ts' for writing
[hls @ 0x14b607af0] Opening '/Users/user/Library/Application Support/jellyfin/cache/transcodes/ae278c66fb6a8a9caca834951109e78c44.ts' for writing
[hls @ 0x14b607af0] Opening '/Users/user/Library/Application Support/jellyfin/cache/transcodes/ae278c66fb6a8a9caca834951109e78c45.ts' for writing
[hls @ 0x14b607af0] Opening '/Users/user/Library/Application Support/jellyfin/cache/transcodes/ae278c66fb6a8a9caca834951109e78c46.ts' for writing
[hls @ 0x14b607af0] Opening '/Users/user/Library/Application Support/jellyfin/cache/transcodes/ae278c66fb6a8a9caca834951109e78c47.ts' for writing
size=N/A time=00:01:38.59 bitrate=N/A speed=65.7x    
[hls @ 0x14b607af0] Opening '/Users/user/Library/Application Support/jellyfin/cache/transcodes/ae278c66fb6a8a9caca834951109e78c48.ts' for writing
[hls @ 0x14b607af0] Opening '/Users/user/Library/Application Support/jellyfin/cache/transcodes/ae278c66fb6a8a9caca834951109e78c49.ts' for writing
[out#0/hls @ 0x600003254900] video:0kB audio:4610kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
size=N/A time=00:01:41.07 bitrate=N/A speed=66.1x

felix920506 commented 4 months ago

I also noticed a few extra things:

the exact same problem happens in Safari
Copying out the segments from the transcodes folder and playing the m3u8 file in VLC doesn't have this problem
On MacOS Safari the issue seems to goes away when I edit the web source code to use hls.js instead of Safari's native HLS implementation
If I disable HLS streaming music through web and make it transcode to a .aac instead, the issue also goes away on MacOS Safari (didn't test iOS Safari on this one).

Chaphasilor commented 4 months ago

I know what you're trying to achieve, I just mentioned the issues in case the transcoding wasn't taking place at all. That doesn't seem to be the case though.

It might be a bug in ffmpeg, that wouldn't be unheard of with regards to HLS transcoding. I'm not surprised VLC can handle it, but sadly that can't be used for playback in our case.

Since I don't have any Apple devices, I'm not able to reproduce the issue myself, and I don't have much experience with transcoding to begin with. Maybe @jmshrv can help out here.

Are there any more details you could share about the media? A demo file would probably help a lot!

felix920506 commented 4 months ago

@Chaphasilor just grab some random music file and convert it to ogg Vorbis with ffmpeg will recreate the issue.

More details:

If I hack web to say other browsers don't support ogg Vorbis and cause it to transcode in both chrome and Firefox the issue doesn't happen.
I've seen some related discussion elsewhere about lower bitrates being unable to properly fill packets when using TS. https://www.wowza.com/community/t/stuttering-audio-on-jwplayer-in-safari-and-ios-using-hls-transport-streams/41395

felix920506 commented 4 months ago

dropping by to add more info. These are recordings of the same source file playing on my phone, one transcoded to HLS AAC in FinAmp and the other directly playing the source file in VLC. The stutters should be quite obvious.

https://github.com/jmshrv/finamp/assets/25688628/539f2397-4000-4739-925b-63ccbe3a4a2e

https://github.com/jmshrv/finamp/assets/25688628/f2ef4957-4772-4f1d-8088-d6d386c3c4bc

Chaphasilor commented 4 months ago

Okay, another question. What happens when you use the beta and download a transcoded version of that track. Transcoded downloads on iOS are limited to mp3 on 10.8, with support for AAC coming with 10.9.

Since regular transcoding always uses AAC afaik, maybe Vorbis->mp3 works better? 🤔

(Also, it's Finamp, not "FinAmp" ^^ Just like it's not "JellyFin" :P)

felix920506 commented 4 months ago

How do I only download one song and not the entire album?

felix920506 commented 3 months ago

Jellyfin-web was able to solve this by using fmp4 container for streaming instead of TS https://github.com/jellyfin/jellyfin-web/pull/5378

jmshrv / finamp

Audio stuttering when transcode playing ogg vorbis on iOS #600