search

bilibili / ijkplayer

Android/iOS video player based on FFmpeg n3.4, with MediaCodec, VideoToolbox support.
GNU General Public License v2.0
32.45k stars 8.11k forks source link

Playing UDP unicast stream with IJKPlayer #1307

Closed bmariesan closed 10 months ago

bmariesan commented 8 years ago

I'm trying to play a video that I'm streaming with ffmpeg as such:

ffmpeg -f dshow -video_size 640x360 -rtbufsize 702000k -framerate 30 -i video="Integrated Camera":audio="Microphone (5- Logitech USB Headset H340)" -r 30 -threads 4 -vcodec libx264 -pix_fmt yuv420p -tune zerolatency -preset ultrafast -f mpegts udp://phone_public_ip:nat_public_port

The device recording the stream is in a different Wi-Fi network than the Android phone using IJKPlayer. The stream works ok when in LAN, however it doesn't between networks. I know that this isn't possible without NAT traversal (UDP hole punching in my case) so I've built a mechanism capable of sending UDP messaged between the device and Android mobile phone.

Unfortunately even though I have the P2P connectivity in place, I cannot seem to find a way to properly play the stream using IJKPlayer. I'm building IJKPlayer with the default config with more codecs.

My Android code:

                    mediaPlayer = new IjkMediaPlayer();

                    AsyncTask<String, Void, String> execute = new P2PConnection().execute("");
                    final String p2pConnectionURL = execute.get();

                    mediaPlayer.setDataSource(getContext(),Uri.parse(p2pConnectionURL));
                    mediaPlayer.setSurface(surface);
                    mediaPlayer.setLooping(false);
                    mediaPlayer.prepareAsync();

                    mediaPlayer.setOption(IjkMediaPlayer.OPT_CATEGORY_FORMAT,"analyzeduration", 2000000);
                    mediaPlayer.setOption(IjkMediaPlayer.OPT_CATEGORY_FORMAT,"fflags", "nobuffer");
                    mediaPlayer.setOption(IjkMediaPlayer.OPT_CATEGORY_FORMAT,"probsize", 4096);

                    mediaPlayer.setOnPreparedListener(new IjkMediaPlayer.OnPreparedListener() {
                        @Override
                        public void onPrepared(IMediaPlayer mp) {
                            if (isVisibleHint) {
                                play();
                            }
                        }
                    });

                    mediaPlayer.setOnCompletionListener(new IjkMediaPlayer.OnCompletionListener() {
                        @Override
                        public void onCompletion(IMediaPlayer mp) {
                            AnimationUtils.fadeIn(llMediaController, 200);
                            ivPlay.setImageResource(R.drawable.play);
                            mediaProgressHandler.removeMessages(SHOW_PROGRESS);
                        }
                    });

On that AsyncTask I'm computing the URL and I've tried the below combinations to form it:

            p2pConnectionURL = "udp://"+privatePhoneIP+":"+random_port+"?localport="+punchedPort;
            p2pConnectionURL = "udp://"+privatePhoneIP+":"+random_port;
            p2pConnectionURL = "udp://@:"+punchedPort;
            p2pConnectionURL = "udp:/"+publicPhoneNatIP+":"+random_port+"?localport="+punchedPort;
            p2pConnectionURL = "udp:/"+publicPhoneNatIP+":"+punchedPort

My IJKPlayer log when trying all of the options above looks like:

05-11 15:34:34.579 27243-27243/io.github D/J4A: J4ALoader: OK: 'tv.danmaku.ijk.media.player.IjkMediaPlayer' loaded
05-11 15:34:34.599 27243-27243/io.github D/IJKMEDIA: IjkMediaPlayer_native_init
05-11 15:34:34.599 27243-27243/io.github D/IJKMEDIA: IjkMediaPlayer_native_setup
05-11 15:34:34.599 27243-27243/io.github I/IJKMEDIA: av_version_info: ff3.0--ijk0.5.0--dev0.4.5--rc11
05-11 15:34:34.599 27243-27243/io.github D/IJKMEDIA: ffpipeline_create_from_android()
05-11 15:34:34.599 27243-27243/io.github D/IJKMEDIA: ijkmp_set_inject_opaque(0x1004aa)
05-11 15:34:34.599 27243-27243/io.github D/IJKMEDIA: ijkmp_set_inject_opaque()=void
05-11 15:34:34.599 27243-27243/io.github D/IJKMEDIA: ijkmp_android_set_mediacodec_select_callback()
05-11 15:34:34.599 27243-27243/io.github D/IJKMEDIA: ffpipeline_set_mediacodec_select_callback
05-11 15:34:34.599 27243-27243/io.github D/IJKMEDIA: ijkmp_android_set_mediacodec_select_callback()=void
05-11 15:34:44.889 27243-27243/io.github D/tv.danmaku.ijk.media.player.IjkMediaPlayer: Couldn't open file on client side, trying server side
05-11 15:34:44.889 27243-27243/io.github D/IJKMEDIA: IjkMediaPlayer_setDataSourceAndHeaders
05-11 15:34:44.889 27243-27243/io.github D/IJKMEDIA: ijkmp_set_data_source(url="udp://@:46329")
05-11 15:34:44.889 27243-27243/io.github D/IJKMEDIA: ijkmp_set_data_source(url="udp://@:46329")=0
05-11 15:34:44.889 27243-27243/io.github D/IJKMEDIA: IjkMediaPlayer_setVideoSurface
05-11 15:34:44.889 27243-27243/io.github D/IJKMEDIA: ijkmp_set_android_surface(surface=0xbe829870)
05-11 15:34:44.889 27243-27243/io.github D/IJKMEDIA: ffpipeline_set_surface()
05-11 15:34:44.889 27243-27243/io.github D/IJKMEDIA: ijkmp_set_android_surface(surface=0xbe829870)=void
05-11 15:34:44.889 27243-27243/io.github D/IJKMEDIA: IjkMediaPlayer_setOptionLong
05-11 15:34:44.889 27243-27243/io.github D/IJKMEDIA: IjkMediaPlayer_setLoopCount
05-11 15:34:44.889 27243-27243/io.github D/IJKMEDIA: IjkMediaPlayer_prepareAsync
05-11 15:34:44.889 27243-27243/io.github D/IJKMEDIA: ijkmp_prepare_async()
05-11 15:34:44.889 27243-27243/io.github I/IJKMEDIA: ===== versions =====
05-11 15:34:44.889 27243-27243/io.github I/IJKMEDIA: FFmpeg       : ff3.0--ijk0.5.0--dev0.4.5--rc11
05-11 15:34:44.889 27243-27243/io.github I/IJKMEDIA: libavutil    : 55.17.103
05-11 15:34:44.889 27243-27243/io.github I/IJKMEDIA: libavcodec   : 57.24.102
05-11 15:34:44.889 27243-27243/io.github I/IJKMEDIA: libavformat  : 57.25.100
05-11 15:34:44.889 27243-27243/io.github I/IJKMEDIA: libswscale   : 4.0.100
05-11 15:34:44.899 27243-27243/io.github I/IJKMEDIA: libswresample: 2.0.101
05-11 15:34:44.899 27243-27243/io.github I/IJKMEDIA: ===== options =====
05-11 15:34:44.899 27243-28499/io.github I/IJKMEDIA: SDL_RunThread: [28499] ff_msg_loop
05-11 15:34:44.899 27243-27243/io.github I/IJKMEDIA: player-opts : loop                         = 1
05-11 15:34:44.899 27243-27243/io.github I/IJKMEDIA: format-opts : ijkinject-opaque             = 1049770
05-11 15:34:44.899 27243-28499/io.github D/IJKMEDIA: message_loop
05-11 15:34:44.899 27243-27243/io.github I/IJKMEDIA: format-opts : ijkapplication               = -1675637248
05-11 15:34:44.899 27243-27243/io.github I/IJKMEDIA: ===================
05-11 15:34:44.899 27243-28500/io.github I/IJKMEDIA: SDL_RunThread: [28500] ff_vout
05-11 15:34:44.899 27243-27243/io.github D/IJKMEDIA: ijkmp_prepare_async()=0
05-11 15:34:44.899 27243-28501/io.github I/IJKMEDIA: SDL_RunThread: [28501] ff_read
05-11 15:34:44.899 27243-27243/io.github D/IJKMEDIA: IjkMediaPlayer_setOptionLong
05-11 15:34:44.909 27243-27243/io.github D/IJKMEDIA: IjkMediaPlayer_setOption
05-11 15:34:44.909 27243-27243/io.github D/IJKMEDIA: IjkMediaPlayer_setOptionLong
05-11 15:34:44.909 27243-28499/io.github D/IJKMEDIA: FFP_MSG_FLUSH:
05-11 15:34:48.649 27243-27243/io.github D/IJKMEDIA: IjkMediaPlayer_start
05-11 15:34:48.649 27243-27243/io.github D/IJKMEDIA: ijkmp_start()
05-11 15:34:48.649 27243-27243/io.github D/IJKMEDIA: ijkmp_start()=-3

At this point the player hangs and doesn't play anything. I've noticed that UDP packets enter the destination router so I'm guessing either I need to build IJKPlayer a bit differently either I'm not receiving the stream properly.

Can you give me any hints or suggestions of what I might be doing wrong?

bbcallen commented 8 years ago

You need call start() only after OnPrepared.

bmariesan commented 8 years ago

I've moved the start method and now only invoking it when the stream is marked as "playable" but still the same result. It works on my local network but fails between networks:

05-12 11:33:50.985 5454-5454/io.github D/J4A: J4ALoader: OK: 'tv.danmaku.ijk.media.player.IjkMediaPlayer' loaded
05-12 11:33:51.005 5454-5454/io.github D/IJKMEDIA: IjkMediaPlayer_native_init
05-12 11:33:51.005 5454-5454/io.github D/IJKMEDIA: IjkMediaPlayer_native_setup
05-12 11:33:51.005 5454-5454/io.github I/IJKMEDIA: av_version_info: ff3.0--ijk0.5.0--dev0.4.5--rc11
05-12 11:33:51.005 5454-5454/io.github D/IJKMEDIA: ffpipeline_create_from_android()
05-12 11:33:51.005 5454-5454/io.github D/IJKMEDIA: ijkmp_set_inject_opaque(0x1004aa)
05-12 11:33:51.005 5454-5454/io.github D/IJKMEDIA: ijkmp_set_inject_opaque()=void
05-12 11:33:51.005 5454-5454/io.github D/IJKMEDIA: ijkmp_android_set_mediacodec_select_callback()
05-12 11:33:51.005 5454-5454/io.github D/IJKMEDIA: ffpipeline_set_mediacodec_select_callback
05-12 11:33:51.005 5454-5454/io.github D/IJKMEDIA: ijkmp_android_set_mediacodec_select_callback()=void
05-12 11:34:01.235 5454-5454/io.github D/tv.danmaku.ijk.media.player.IjkMediaPlayer: Couldn't open file on client side, trying server side
05-12 11:34:01.235 5454-5454/io.github D/IJKMEDIA: IjkMediaPlayer_setDataSourceAndHeaders
05-12 11:34:01.235 5454-5454/io.github D/IJKMEDIA: ijkmp_set_data_source(url="udp://78.97.202.99:42203")
05-12 11:34:01.235 5454-5454/io.github D/IJKMEDIA: ijkmp_set_data_source(url="udp://78.97.202.99:42203")=0
05-12 11:34:01.235 5454-5454/io.github D/IJKMEDIA: IjkMediaPlayer_setVideoSurface
05-12 11:34:01.235 5454-5454/io.github D/IJKMEDIA: ijkmp_set_android_surface(surface=0xbe829870)
05-12 11:34:01.235 5454-5454/io.github D/IJKMEDIA: ffpipeline_set_surface()
05-12 11:34:01.235 5454-5454/io.github D/IJKMEDIA: ijkmp_set_android_surface(surface=0xbe829870)=void
05-12 11:34:01.235 5454-5454/io.github D/IJKMEDIA: IjkMediaPlayer_setOptionLong
05-12 11:34:01.235 5454-5454/io.github D/IJKMEDIA: IjkMediaPlayer_setLoopCount
05-12 11:34:01.235 5454-5454/io.github D/IJKMEDIA: IjkMediaPlayer_prepareAsync
05-12 11:34:01.235 5454-5454/io.github D/IJKMEDIA: ijkmp_prepare_async()
05-12 11:34:01.235 5454-5454/io.github I/IJKMEDIA: ===== versions =====
05-12 11:34:01.235 5454-5454/io.github I/IJKMEDIA: FFmpeg       : ff3.0--ijk0.5.0--dev0.4.5--rc11
05-12 11:34:01.235 5454-5454/io.github I/IJKMEDIA: libavutil    : 55.17.103
05-12 11:34:01.235 5454-5454/io.github I/IJKMEDIA: libavcodec   : 57.24.102
05-12 11:34:01.235 5454-5928/io.github I/IJKMEDIA: SDL_RunThread: [5928] ff_msg_loop
05-12 11:34:01.235 5454-5454/io.github I/IJKMEDIA: libavformat  : 57.25.100
05-12 11:34:01.235 5454-5928/io.github D/IJKMEDIA: message_loop
05-12 11:34:01.235 5454-5454/io.github I/IJKMEDIA: libswscale   : 4.0.100
05-12 11:34:01.235 5454-5454/io.github I/IJKMEDIA: libswresample: 2.0.101
05-12 11:34:01.235 5454-5454/io.github I/IJKMEDIA: ===== options =====
05-12 11:34:01.235 5454-5454/io.github I/IJKMEDIA: player-opts : loop                         = 1
05-12 11:34:01.235 5454-5454/io.github I/IJKMEDIA: format-opts : ijkinject-opaque             = 1049770
05-12 11:34:01.235 5454-5454/io.github I/IJKMEDIA: format-opts : ijkapplication               = -1718409120
05-12 11:34:01.235 5454-5454/io.github I/IJKMEDIA: ===================
05-12 11:34:01.245 5454-5929/io.github I/IJKMEDIA: SDL_RunThread: [5929] ff_vout
05-12 11:34:01.245 5454-5454/io.github D/IJKMEDIA: ijkmp_prepare_async()=0
05-12 11:34:01.245 5454-5928/io.github D/IJKMEDIA: FFP_MSG_FLUSH:
05-12 11:34:01.245 5454-5930/io.github I/IJKMEDIA: SDL_RunThread: [5930] ff_read
05-12 11:34:01.245 5454-5454/io.github D/IJKMEDIA: IjkMediaPlayer_setOptionLong
05-12 11:34:01.245 5454-5454/io.github D/IJKMEDIA: IjkMediaPlayer_setOption
05-12 11:34:01.245 5454-5454/io.github D/IJKMEDIA: IjkMediaPlayer_setOptionLong
05-12 11:34:12.815 5454-5454/io.github D/IJKMEDIA: Is stream playable:true
05-12 11:34:12.815 5454-5454/io.github D/IJKMEDIA: Is stream playing:false
05-12 11:34:12.815 5454-5454/io.github D/IJKMEDIA: IjkMediaPlayer_getLoopCount
05-12 11:34:12.815 5454-5454/io.github D/IJKMEDIA: Is stream isLooping:false
05-12 11:34:12.815 5454-5454/io.github D/IJKMEDIA: IjkMediaPlayer_start
05-12 11:34:12.815 5454-5454/io.github D/IJKMEDIA: ijkmp_start()
05-12 11:34:12.825 5454-5454/io.github D/IJKMEDIA: ijkmp_start()=-3
bbcallen commented 8 years ago

observe the notification IJKMPMediaPlaybackIsPreparedToPlayDidChangeNotification or set mp.shouldAutoplay = YES

bmariesan commented 8 years ago

I don't have that much experience with native code so I hope I did the right changes:

In class IJKAVMoviePlayerController.m I've changed the mp.shouldAutoplay from NO to YES, then did a clean build for ffmpeg and re-compiled ijkplayer. I still get the exact same output as above with the stream still not working between networks

bbcallen commented 8 years ago

Sorry, thought you were on iOS. For android, you should call start after OnPreparedListener being called.

bmariesan commented 8 years ago

Sorry I forgot to mention that from the beginning :) It seems that as soon as I try to stream outside my current network, so to stream between networks, the OnPreparedListener is never called. Any idea why that might happen?

personshelldon commented 8 years ago

I have some information about multicast play with this player and ffmpeg at all. I'm writing player for IOS and Android based on ijk and thanks for it (subtitles also works correct, thanks) and I faced with this probem. I learned everything that I found about ffmpeg, ijk and multicast and what I found: ffmpeg supports playing multicast from the box, you need only compile it with support rtp/udp protocols. To compile ijk with this support You need use default modules file in config folder. But there is another "but" - many Android devices does not support multicast at all, it is locked by the kernel of the device for service (OTA updates) and battery saving perposes (this is answer from Google). For example, I tested multicast on Nexus 6P, Nexus 5, Prestigio and HTC One m7 and it works only on HTC One. On other devices it hangs on av_open_input, because the device can not receive multicast packets at all. On IOS I tested ijk on last iPad and it works normally. For Android there is only one thing that we can do - convert multicast to unicast, it is also working normally.

bbcallen commented 8 years ago

Sad to hear that.

discoy commented 7 years ago

@don11995 tks your answer,You have mentioned "convert multicast to unicast, it is also working normally."but how to convert?could you give some sample code?

personshelldon commented 7 years ago

@discoy, You need to set up UDPProxy on Your router or switch. I did not find any other solution for this problem (only recompile the kernel with support of the multicast, but it is not a valid solution for me).