[ios][video_player] pip - picture in picture

[p30arena]

Use case

I have a customer asking for PIP when the app is playing a video and the user wants multi-tasking.

For Android I had to edit the plugin, and use this line of code: getActivity().enterPictureInPictureMode(); https://developer.android.com/guide/topics/ui/picture-in-picture#java

But for iOS, it is a different story, iOS does not support PIP in UIWindow or UIView, it is supported only in a Custom Player. We need a custom view for that. https://developer.apple.com/documentation/avkit/adopting_picture_in_picture_in_a_custom_player https://developer.apple.com/documentation/avfoundation/avplayerlayer

The video_player plugin is using Texture to show the video output.

Proposal

I need a way to customize the UIView created by the engine for the texture and add AVPlayerLayer support to it. Any suggestions on how can I contribute this feature to the plugin?

[aytunch]

Sorry if I am pointing the obvious but is there anything wrong with using a Stack with 2 video_players as its children?

[p30arena]

@bparrishMines @xster @gaaclarke @amirh

[gaaclarke]

I need a way to customize the UIView created by the engine for the texture and add AVPlayerLayer support to it. Any suggestions on how can I contribute this feature to the plugin?

You don't need to "customize" the view per-say. Why not just get access to the root view controller (FlutterViewController) and add your own layer to that view, or add another view to it.

Something like:

UIViewController* vc = [[[UIApplication sharedApplication] keyWindow] rootViewController];
MyPipView* pipView = [[MyPipView alloc] init];
[vc.view addSubview:pipView];

[p30arena]

Thank you @gaaclarke I think this doesn't worth a try, PiP is only supported on the iPad.

[StefanLobbenmeier]

@p30arena please reopen, PiP is supported from iOS 14

[p30arena]

@StefanLobbenmeier https://developer.apple.com/documentation/avkit/avplayerviewcontroller

Adopting Picture in Picture Playback Use AVPlayerViewController to provide Picture in Picture (PiP) playback on supported iPad models. PiP playback lets users minimize your video player to a small floating window so they can perform other activities in the primary application or even in another application. This brings a new level of multitasking capabilities to the iPad allowing users to continue playback while performing other activities on their devices.

Use AVPlayerViewController to provide Picture in Picture (PiP) playback on supported iPad models.

[StefanLobbenmeier]

This is the announcement at wwdc: https://youtu.be/GEZhD3J89ZE?t=11m44s . I will check the developer Information for iOS 14 Beta if they also mention it there, but I can confirm on my device (iPhone 7 iOS 14 beta 2) pip works

[StefanLobbenmeier]

The best I could find in the documentation is https://developer.apple.com/documentation/avkit/adopting_picture_in_picture_in_a_custom_player Which mentions neither iPad nor iPhone, but that’s just a subpage of your page so I think o am not looking at the correct place

[alexmarkley]

I'm also affected by this. My users will expect to be able to continue playing videos while the app is backgrounded wherever the platform supports it.

[thanhit93]

Please refer to my github: https://github.com/thanhit93/plugins/tree/master/packages/video_player/video_player

[StefanLobbenmeier]

@thanhit93 thanks for sharing - have you also tested it on iPad? I am a bit unsure about the conditions in your source code / if they only work on iPhone or also iPad

[thanhit93]

Hello @StefanLobbenmeier This plugin I just developed. I only tested on iphone (iOS14), ipad hasn't been tested. I will check on the iPad and fix the error as soon as possible. I also look forward to contributing from anyone to make the plugin even better.

[flymzero]

@thanhit93 This looks great. It would be better if it could be made into a plug-in package. Thank you

[anychhoice]

any updates?

[DesmondFox]

very useful feature! Any updates?

[AnmolSethi]

Please refer to my github: https://github.com/thanhit93/plugins/tree/master/packages/video_player/video_player

@thanhit93 How do I use it in my project?

[thanhit93]

Please refer to my github: https://github.com/thanhit93/plugins/tree/master/packages/video_player/video_player

@thanhit93 How do I use it in my project?

update pubspec.yaml file:

video_player:
   git:
      url: https://github.com/thanhit93/plugins.git
      ref: master
      path: packages/video_player/video_player 

import 'dart:io'; Show PIP:

onPIPTap: () async {
         if (!Platform.isIOS) return;
         await videoController.pause();
         final MediaQueryData data = MediaQuery.of(context);
         EdgeInsets paddingSafeArea = data.padding;
         double widthScreen = data.size.width;
         videoController ?.setPIP(true, left: 0, top: paddingSafeArea.top, width: widthScreen, height: 9 * widthScreen / 16);
         videoController.play();
}

Hide PIP:

if (Platform.isIOS) videoController.setPIP(false);

[IlyaMax]

@thanhit93 why not to contribute with this?

[vanlooverenkoen]

Good news!!! We had to implement Picture in Picture mode for one of our clients. We started from the implementation that @thanhit93 did. The package was so outdated that we had to start over again.

Without any knowledge of objective-c, we started the implementation.

For now everything is working in the example project. (expect swipe to close app without picture in picture manually opened, this is something I am investigating)

When testing the example project you should replace all hosted dependencies of video_player.... to path dependencies.

video_player:
   git:
      url: https://github.com/icapps/plugins.git
      ref: feature/ios-pip
      path: packages/video_player/video_player 

We do want to push this to remote. But it will probably not get merged if we don't have iOS tests. We don't have any experience so we hope we can find somebody that can help us with that. (I had no luck with my previous google_maps pull request 😞 )

https://user-images.githubusercontent.com/21172855/185163421-f4138ddd-e08c-4563-8bbc-174cd996106f.MP4

[vanlooverenkoen]

Thanks to (https://stackoverflow.com/a/67273380/5115254) I found that you still had to set an option if you want your app to automatically go to pip or not. I will add this as well to complete the full PiP implementation.

[AnmolSethi]

@vanlooverenkoen I can help you with iOS tests!

[vanlooverenkoen]

@AnmolSethi 🙏 awesome. Could you maybe already review the pull request and maybe give some suggestions where or what should be tested more, and how I should test them?

[harryandroiddev]

add this feature to the package also.

[harryandroiddev]

Good news!!! We had to implement Picture in Picture mode for one of our clients. We started from the implementation that @thanhit93 did. The package was so outdated that we had to start over again.

Without any knowledge of objective-c, we started the implementation.

For now everything is working in the example project. (expect swipe to close app without picture in picture manually opened, this is something I am investigating)

When testing the example project you should replace all hosted dependencies of video_player.... to path dependencies.

video_player:
   git:
      url: https://github.com/icapps/plugins.git
      ref: feature/ios-pip
      path: packages/video_player/video_player 

We do want to push this to remote. But it will probably not get merged if we don't have iOS tests. We don't have any experience so we hope we can find somebody that can help us with that. (I had no luck with my previous google_maps pull request 😞 )

video-player-pip-demo-ios.MP4

Try correcting the name to the name of an existing method, or defining a method named 'stopPictureInPicture'. await _videoPlayerPlatform.stopPictureInPicture(_textureId); ^^^^^^^^^^^^^^^^^^^^

[sshsato]

add this

dependency_overrides:
  video_player_platform_interface:
    git:
      url: https://github.com/icapps/plugins.git
      ref: feature/ios-pip
      path: packages/video_player/video_player_platform_interface
  video_player_avfoundation:
    git:
      url: https://github.com/icapps/plugins.git
      ref: feature/ios-pip
      path: packages/video_player/video_player_avfoundation
  video_player_android:
    git:
      url: https://github.com/icapps/plugins.git
      ref: feature/ios-pip
      path: packages/video_player/video_player_android

[t0uh33d]

Hey @vanlooverenkoen First of all, i would like to thank you for your contribution towards the PiP mode, It reduced a lot of work since i had to implement this feature in an application. It works well, but the problem is PiP mode is not getting triggered when transitioning to home screen even after setting the canStartPictureInPictureAutomaticallyFromInline to true.

I have been trying various thing, but still there is no luck. Were you able to trigger the PiP mode automatically while the user transitioned to the home screen?

[vanlooverenkoen]

Yes for me that worked perfectly. as you can see here: https://github.com/icapps/plugins/blob/565012be9017a552296e4169537e5c0098b6f288/packages/video_player/video_player/doc/demo_pip_iphone.gif

Please check out my pull request to run the example first and see what is missing in your code.

[t0uh33d]

Thank you the quick response!! I went through the code and found that i had not used the setPictureInPictureOverlayRect method. Once i did that, it worked perfectly for me.

[vanlooverenkoen]

Yes, it is important to use the setPictureInPictureOverlayRect because iOS needs to draw a native view where it should be shown. Otherwise it is not possible to automatically animate the video_player with the pip

[t0uh33d]

I was trying everything but could not make it work, so i created an ios project and implemented the pip mode there, it was working perfectly, but the same code wasn't working in Flutter, i was so confused. But i think i got it now, when we attach the AVPictureInPictureController to a native component i guess it draws the required values automatically and hence it was working.

[vanlooverenkoen]

Hmm do you still have the code for that? Maybe it would help me to simplify the code for the pip as well

[vanlooverenkoen]

But from my testing I wasn't able to get it working. But again, I am not an iOS dev so maybe I am missing something. Tonight I will be opening my pull request again :D

[t0uh33d]

https://github.com/developerinsider/Picture-in-Picture-iOS-14-Demo I referred this repository, hope it helps you out.

[vanlooverenkoen]

Oeeeef!! Nice! :D thanks alot!

[t0uh33d]

And also, while the pip is getting enabled automatically now, right after going into the pip mode, the video is getting paused. Whereas in the example project which you have provided, it continues playing, Have you explicitly handled it, or do you think it might have to do something with my code. PS : I have used Flick Video Player, so i am having doubts if that is causing this behaviour.

[vanlooverenkoen]

It has been a while since I implemented it. I will double check this evening

[t0uh33d]

Thanks a lot for your help :) And please don't mind it, i will have a look and try to solve it myself, at least i can do this :P

[krishaajtak]

@vanlooverenkoen I also want to help. Can you just mention a starting point so I can build for it?

[vanlooverenkoen]

Check out the pull request I made and run the example to start

[KavyashreeBasavesh]

@vanlooverenkoen @krishaajtak Hello, Aplogize for writing over hear. Just wanted to check, if you people can help with the issue we are facing. We are using a framework of WebRTC to build an video call application. Client is asking us to implement the PIP in iOS. Is there any solution or code snippet which is helpful for us to implement the same using [AVPlayerViewController] and we are developing using swift language.

[vanlooverenkoen]

You can checkout the PiP branch (pr) it is working for us in production for almost 2 years already

[KavyashreeBasavesh]

@vanlooverenkoen Thanks for your response in short time. https://developer.apple.com/documentation/avkit/adopting_picture_in_picture_for_video_calls We are looking PIP feature for video calls which is being developed using webRTC.(RTCMTLvideoview). Please let us know, if you have any idea or way to implement this.

[KavyashreeBasavesh]

Hi @vanlooverenkoen , Did you get any chance to check on this? It will be very helpful, if you share your insight.

[vanlooverenkoen]

You can take a look at the PR I have open, that contains the full implementation for PIP using video_player https://github.com/flutter/packages/pull/3500 maybe you can find something there

[Sarandjj]

I want to use pip mode in video player with play push button in flotation