search

tanersener / mobile-ffmpeg

FFmpeg for Android, iOS and tvOS. Not maintained anymore. Superseded by FFmpegKit.
https://tanersener.github.io/mobile-ffmpeg
GNU General Public License v3.0
3.85k stars 787 forks source link

mobile-ffmpeg not streaming on iOS #655

Closed dev-manager-uk closed 3 years ago

dev-manager-uk commented 3 years ago

Description We have built an iOS sample application with mobile-ffmeg: https://github.com/dev-manager-uk/ios-ffmpeg

Expected behavior Play tracks when streaming through mobile-ffmpeg

Current behavior When trying to stream using mobile-ffmpeg we get "Operation Stopped. The server is not correctly configured"

Logs some : Error Domain=AVFoundationErrorDomain Code=-11850 "Operation Stopped" UserInfo={NSLocalizedFailureReason=The server is not correctly configured., NSLocalizedDescription=Operation Stopped, NSUnderlyingError=0x6000032a01e0 {Error Domain=NSOSStatusErrorDomain Code=-12939 "(null)"}}

+"The HTTP server sending the media resource is not configured as expected. This might mean that the server does not support byte range requests."

To Reproduce

Any help greatly appreciated :-)

tanersener commented 3 years ago

Can you please give details about what this application is doing, what are the ffmpeg commands used?

dev-manager-uk commented 3 years ago

Yes, it's trying to stream an audio file from either local path or gDrive, through FFMPEG, to iOS Audio player

dev-manager-uk commented 3 years ago

-i pathToTrack/URL -vn -strict -2 -acodec pcm_u8 -f wav -listen 1 -seekable 1 http://localhost:8090/restream.wav

tanersener commented 3 years ago

Your command sends the stream to a server at port 8090. Is your player listening on that port?

dev-manager-uk commented 3 years ago

It does, here's the log:

2021-01-16 08:25:50.331496+0000 DemoFFMEG[86564:1300779] Loaded mobile-ffmpeg-full-x86_64-4.4-20200725 !ffmpeg version v4.4-dev-416 ! Copyright (c) 2000-2020 the FFmpeg developers !

! built with Apple clang version 11.0.3 (clang-1103.0.32.62)

! configuration: --sysroot=/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator13.6.sdk --prefix=/Users/taner/Projects/mobile-ffmpeg/prebuilt/ios-x86_64/ffmpeg --enable-version3 --arch=x86_64 --cpu=x86_64 --target-os=darwin --ar=/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ar --cc=clang --cxx=clang++ --as='clang -arch x86_64 -target x86_64-ios-darwin -march=x86-64 -msse4.2 -mpopcnt -m64 -mtune=intel -DMOBILE_FFMPEG_X86_64 -Wno-unused-function -Wno-deprecated-declarations -fstrict-aliasing -DIOS -DMOBILE_FFMPEG_BUILD_DATE=20200725 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator13.6.sdk -O2 -mios-simulator-version-min=12.1 -I/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator13.6.sdk/usr/include' --ranlib=/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib --strip=/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/strip --disable-neon --enable-cross-compile --enable-pic --disable-asm --enable-inline-asm --enable-optimizations --enable-swscale --enable-static --disable-shared --enable-small --disable-v4l2-m2m --disable-outdev=v4l2 --disable-outdev=fbdev --disable-outdev=audiotoolbox --disable-indev=v4l2 --disable-indev=fbdev --disable-openssl --disable-xmm-clobber-test --disable-debug --disable-neon-clobber-test --disable-programs --disable-postproc --disable-doc --disable-htmlpages --disable-manpages --disable-podpages --disable-txtpages --disable-sndio --disable-schannel --disable-securetransport --disable-xlib --disable-cuda --disable-cuvid --disable-nvenc --disable-vaapi --disable-vdpau --disable-appkit --disable-alsa --disable-cuda --disable-cuvid --disable-nvenc --disable-vaapi --disable-vdpau --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-gmp --enable-gnutls --enable-libmp3lame --enable-libass --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libxml2 --enable-libopencore-amrnb --enable-libshine --enable-libspeex --enable-libwavpack --enable-libkvazaar --enable-libilbc --enable-libopus --enable-libsnappy --enable-libsoxr --enable-libaom --enable-libtwolame --disable-sdl2 --enable-libvo-amrwbenc --enable-zlib --enable-audiotoolbox --enable-bzlib --enable-videotoolbox --enable-avfoundation --enable-iconv --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-gmp --enable-gnutls --enable-libmp3lame --enable-libass --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libxml2 --enable-libopencore-amrnb --enable-libshine --enable-libspeex --enable-libwavpack --enable-libkvazaar --enable-libilbc --enable-libopus --enable-libsnappy --enable-libsoxr --enable-libaom --enable-libtwolame --disable-sdl2 --enable-libvo-amrwbenc --enable-zlib --enable-audiotoolbox --enable-bzlib --enable-videotoolbox --enable-avfoundation --enable-iconv

! libavutil 56. 55.100 / 56. 55.100

! libavcodec 58. 96.100 / 58. 96.100

! libavformat 58. 48.100 / 58. 48.100

! libavdevice 58. 11.101 / 58. 11.101

! libavfilter 7. 87.100 / 7. 87.100

! libswscale 5. 8.100 / 5. 8.100

! libswresample 3. 8.100 / 3. 8.100

!Estimating duration from bitrate, this may be inaccurate

!Input #0, mp3, from '/Users/user1/Library/Developer/CoreSimulator/Devices/39389A4A-D32E-4AB7-9CDD-995E8D257FB7/data/Containers/Bundle/Application/99C071E9-5418-47DC-9BB7-752C9FA84499/DemoFFMEG.app/Daft Punk - Get Lucky (Joe Maz Remix) (Extended).mp3':

! Metadata:

! title : !Get Lucky !

! artist : !Daft Punk !

! genre : !Electronic !

! TIT1 : !%METADATA% !

! publisher : !BPM Supreme !

! composer : !%METADATA% !

! TKEY : !10A !

! TBPM : !126 !

! comment : !10A !

! copyright : !%METADATA% !

! TOPE : !%METADATA% !

! encoded_by : !%METADATA% !

! performer : !%METADATA% !

! TEXT : !%METADATA% !

! album_artist : !%METADATA% !

! TIT3 : !%METADATA% !

! EnergyLevel : !8 !

! TPE4 : !Joe Maz Remix !

! Duration: !00:05:27.81 !, start: !0.000000 !, bitrate: !335 kb/s !

! Stream #0:0 !: Audio: mp3, 44100 Hz, stereo, fltp, 320 kb/s !

! Stream #0:1 !: Video: mjpeg, yuvj444p(pc, bt470bg/unknown/unknown), 900x900 [SAR 150:150 DAR 1:1] !, !90k tbr, !90k tbn, !90k tbc ! (attached pic) !

! Metadata:

! comment : !Other !

2021-01-16 08:25:51.835821+0000 DemoFFMEG[86564:1305442] [] nw_protocol_get_quic_image_block_invoke dlopen libquic failed !Stream mapping:

! Stream #0:0 -> #0:0 ! (mp3 (mp3float) -> pcm_u8 (native)) !

!Press [q] to stop, [?] for help

!Output #0, wav, to 'http://localhost:8090/restream.wav':

! Metadata:

! INAM : !Get Lucky !

! IART : !Daft Punk !

! IGNR : !Electronic !

! TIT1 : !%METADATA% !

! publisher : !BPM Supreme !

! composer : !%METADATA% !

! TKEY : !10A !

! TBPM : !126 !

! ICMT : !10A !

! ICOP : !%METADATA% !

! TOPE : !%METADATA% !

! ITCH : !%METADATA% !

! performer : !%METADATA% !

! TEXT : !%METADATA% !

! album_artist : !%METADATA% !

! TIT3 : !%METADATA% !

! EnergyLevel : !8 !

! TPE4 : !Joe Maz Remix !

! ISFT : !Lavf58.48.100 !

! Stream #0:0 !: Audio: pcm_u8 ([1][0][0][0] / 0x0001), 44100 Hz, stereo, u8, 705 kb/s !

! Metadata:

! encoder : !Lavc58.96.100 pcm_u8 !

????FAILED ???? Error Domain=AVFoundationErrorDomain Code=-11850 "Operation Stopped" UserInfo={NSLocalizedFailureReason=The server is not correctly configured., NSLocalizedDescription=Operation Stopped, NSUnderlyingError=0x600003bcc4b0 {Error Domain=NSOSStatusErrorDomain Code=-12939 "(null)"}} !av_interleaved_write_frame(): Broken pipe

!Connection to tcp://localhost:8090 failed: Connection refused

!Connection to tcp://localhost:8090 failed: Connection refused

!Connection to tcp://localhost:8090 failed: Connection refused

!Error writing trailer of http://localhost:8090/restream.wav: Broken pipe

!size= 594kB time=00:00:06.92 bitrate= 703.2kbits/s speed= 62x

!video:0kB audio:596kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: !unknown !

!URL read error: Connection reset by peer

!Conversion failed!

3001 1

dev-manager-uk commented 3 years ago

The sample app is useful for debugging. All it does is play the track through ffmpeg:

sample
dev-manager-uk commented 3 years ago

Seems to be related to:

when AVPlayerItem receive a video URL , it do the following task:

Send a bytes request HTTP Request, and range = 0 -1
If the response code is 206 and return 1 bytes data, It do the 3th task, if not, AVErrorServerIncorrectlyConfigured error occurred.
continue send other HTTP Request, to download segment of All duration. and the response of VideoData code must be 206

from: https://stackoverflow.com/a/42244954/3693688

tanersener commented 3 years ago

When I run your sample app I see the same errors.

!Connection to tcp://localhost:8090 failed: Connection refused

!Connection to tcp://localhost:8090 failed: Connection refused

!Connection to tcp://localhost:8090 failed: Connection refused

!Error writing trailer of http://localhost:8090/restream.wav: Broken pipe

They're not hard to understand. These are http errors and tell that no one is listening on port 8090.

Playing a stream and serving a stream are two different things. Yes, it seems like AVPlayer supports playing a stream but does not support serving a stream. I also searched this on google and couldn't find anything about it. Can you show me a link or something that proves AVPlayer supports this?

dev-manager-uk commented 3 years ago

Useful that you were able to reproduce!

We're using AVUrlAsset

Here's the code we're using to call the URL:


    func logCallback( executionId: Int,  level: Int32, _ message: String!) {
        print("!" + message)
        if message.range(of: "Input #0") != nil {
            let url = URL(string: FFMPEG_RESTREAM_ADDRESS)!
            asset = AVURLAsset(url: url)

            // Create a new AVPlayerItem with the asset and an
            playerItem = AVPlayerItem(asset: asset)

            // Register as an observer of the player item's status property
            playerItem.addObserver(self,
                                   forKeyPath: #keyPath(AVPlayerItem.status),
                                   options: [.old, .new],
                                   context: &playerItemContext)

            // Associate the player item with the player
            player = AVPlayer(playerItem: playerItem)
        }
e
dev-manager-uk commented 3 years ago

It seems to be related to this StackOverflow post

when AVPlayerItem receive a video URL , it do the following task:

    Send a bytes request HTTP Request, and range = 0 -1
    If the response code is 206 and return 1 bytes data, It do the 3th task, if not, AVErrorServerIncorrectlyConfigured error occurred.
    continue send other HTTP Request, to download segment of All duration. and the response of VideoData code must be 206

In my situation , when send range[0-1] HTTP request, the server side give me a 200 OK response, So error occurred.

FFMPEG gives a 200 response, but AVPlayer requires a 206 response

We've now replicated using FFMPEG in terminal, bypassing ffmpeg-mobile, and get the same result.

So I guess it seems that the AVPlayer range request for = 0 -1 is incompatible with FFMPEG stream which returns 200 instead of 206 which AVPlayer requires?

Which means it's perhaps not an ffmpeg-mobile issue but an ffmpeg issue?

Apple description for this error

tanersener commented 3 years ago

Which means it's perhaps not an ffmpeg-mobile issue but an ffmpeg issue?

Yeah, it is about ffmpeg but I'm not exactly sure that this is an issue.

FFMPEG gives a 200 response, but AVPlayer requires a 206 response

Which log message shows this?

dev-manager-uk commented 3 years ago

Here's a wireshark log that shows the 200 response for bytes 0-1:

GET /file.wav HTTP/1.1
Host: localhost:1234
X-Playback-Session-Id: F72F1139-6F4C-4A22-B334-407672045A86
Range: bytes=0-1
Accept: */*
User-Agent: AppleCoreMedia/1.0.0.18C61 (iPhone; U; CPU OS 14_3 like Mac OS X; en_us)
Accept-Language: en-us
Accept-Encoding: identity
Connection: keep-alive

HTTP/1.1 200 OK
Content-Type: application/octet-stream
Transfer-Encoding: chunked
dev-manager-uk commented 3 years ago

Is there any way to force ffmpeg to respond with a 206 here?

tanersener commented 3 years ago

Well, it seems like you are using the experimental http server of ffmpeg, which is enabled by -listen 1 option. I think this is why ffmpeg returns 200.

Unfortunately, I'm not familiar with this feature of ffmpeg. I don't know how you can change its behaviour. https://ffmpeg.org/ffmpeg-protocols.html#http lists some options you can try. But I don't know if any of them can fix this error. I'll suggest asking this to ffmpeg developers on ffmpeg-user mailing list.

dev-manager-uk commented 3 years ago

ok, I'll do that. Thank you for your help!

Alanko5 commented 3 years ago

@dev-manager-uk - Please Share me your working code?

dev-manager-uk commented 3 years ago

@dev-manager-uk - Please Share me your working code?

Sure thing, all here: https://github.com/dev-manager-uk/ios-ffmpeg

I'm still trying to solve this one!

dev-manager-uk commented 3 years ago

Interesting, thank you. How would you then get ffmpeg to stream through this swift server?

Alanko5 commented 3 years ago

Have you seen this?

https://developer.apple.com/videos/play/wwdc2020/10011/ https://developer.apple.com/documentation/avfoundation/avassetreader

dev-manager-uk commented 3 years ago

Very, very, very interesting. I hadn't seen that. Thank you @Alanko5! I will investigate further...

github-actions[bot] commented 3 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.