Issues with HLS & HE-AAC

[DiegoMax]

Hello there, im having some issues with HLS when using HE-AAC.

The SBR part of the HE-AAC stream seems to "drop" for a few milliseconds every time a new segment is loaded by the player.

Example stream here:

http://54.172.167.126/hls/onair/index.m3u8

This is how i am building the HE-AAC stream: find /var/music -name "*.mp3" | sort -R | xargs -I "$" \ ffmpeg -re -i "$" \ -c:a libfdk_aac -profile:a aac_he_v2 -b:a 48k -f flv rtmp://127.0.0.1:1935/hls/onair \ -af compand=.3|.3:1|1:-90/-60|-60/-40|-40/-30|-20/-20:6:0:-90:0.2

And this is my config application hls { live on; hls on; hls_fragment 10s; hls_path /tmp/hls; }

I have tried different values for hls_fragment but it really doesnt fix the issue.

Any ideas of what could be causing this???

Thanks!

[andrew867]

I just noticed this exact issue today when playing around with things.

This is my ffmpeg encoding command line:

ffmpeg -loglevel verbose -re -i http://209.95.50.189:8025/ -vn -c:a libfdk_aac -movflags +faststart -profile:a aac_he_v2 -f flv rtmp://localhost:1935/hls/movie

My HLS config:

        application hls {                                                                                                                                    
            live on;
            hls on;
            hls_fragment 15s;
            hls_playlist_length 2m;
            hls_sync 100ms;
            hls_path /tmp/hls;
        }   

[andrew867]

I'm going to recompile ffmpeg with libaacplus to see if that makes any difference.

[misiek08]

You are using audio-only HLS and it was problematic from the beginning. Did you try the same stream with generating black video?

[andrew867]

No I haven't tried black video, I will try that after ffmpeg finishes recompiling with libaacplus support.

[andrew867]

When using libaacplus nginx-rtmp-module recognizes the stream as AAC-LC but Quicktime X recognizes it as HE-AAC v1 and plays fine. VLC will not play the raw TS files, nor will the HLS stream work. It also works fine on an iPhone 5s with IOS 9.1.

ffmpeg -loglevel verbose -re -i http://209.95.50.189:8025/ -vn -c:a libaacplus -movflags +faststart -b:a 48k  -f flv rtmp://localhost:1935/hls/movie

ffmpeg version 2.8.2 Copyright (c) 2000-2015 the FFmpeg developers
  built with gcc 4.7 (Debian 4.7.2-5)
  configuration: --prefix='“/usr”' --extra-cflags=-I/root/ffmpeg_build/include --extra-ldflags=-L/root/ffmpeg_build/lib --bindir=/usr/bin --extra-libs=-ldl 
-enable-gpl --enable-libass --enable-libfdk-aac --enable-libfreetype --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-lib
px --enable-libx264 --enable-nonfree --enable-libaacplus --enable-static
  libavutil      54. 31.100 / 54. 31.100
  libavcodec     56. 60.100 / 56. 60.100
  libavformat    56. 40.101 / 56. 40.101
  libavdevice    56.  4.100 / 56.  4.100
  libavfilter     5. 40.101 /  5. 40.101
  libswscale      3.  1.101 /  3.  1.101
  libswresample   1.  2.101 /  1.  2.101
  libpostproc    53.  3.100 / 53.  3.100

I'll try libfdk_aac and black video now.

[andrew867]

So with 'black' video both VLC and Quicktime play the stream but the SBR breakup doesn't occur when using VLC.

I had to use a component video mode so the video will be green but it serves the same purpose as black.

ffmpeg -loglevel verbose -re -s 640x480 -f rawvideo -pix_fmt yuv420p -r 29.97 -i /dev/zero -i http://209.95.50.189:8025/ -c:a libfdk_aac -movflags +faststart -profile:a aac_he_v2 -profile:v baseline -tune stillimage -c:v libx264  -f flv rtmp://localhost:1935/hls/movie

When using an audio only stream VLC recognizes that the MPEG program table has both video and audio streams but no content in the video stream. The issue might be in the RTMP to TS remux.

Media info screens:

VLC with 'black' video: https://www.dropbox.com/s/6wiyeh296xn9cbs/Screenshot%202015-11-22%2015.19.32.png?dl=0

VLC with audio only: https://www.dropbox.com/s/gzl53q9srwav3a8/Screenshot%202015-11-22%2015.21.26.png?dl=0

Quicktime with 'black' video: https://www.dropbox.com/s/u3ysw0bohexxbi5/Screenshot%202015-11-22%2015.20.00.png?dl=0

Quicktime with audio only: https://www.dropbox.com/s/4qd6dtae2e20hvp/Screenshot%202015-11-22%2015.21.15.png?dl=0

[andrew867]

Somehow I finally got VLC playing the libfdk_aac audio only stream and it breaks up between TS files in a similar way as Quicktime, it's more noticeable as a breakup of the complete stream and not just the SBR.

Do you think this has to do with the TS not having a valid timing stream as the video stream doesn't exist but the program table is reporting it does?

[andrew867]

As I mentioned above when looking at the MPEG TS that is created from the rtmp-module it is not formed correctly for an audio only source. The best option for this is to scrap the static TS remuxing code (as it contains a bunch of byte arrays for the TS file headers and other data) and use libav functions (https://github.com/libav/libav/blob/master/libavformat/mpegts.c) or ffmpeg to do it right and create a proper stream.