[Enhancement] Cross platform media player

Askar938 commented 3 years ago

Summary

Are there any plans for a cross platform media player to support playing audio and video? All features supported by Baseflow/XamarinMediaManager should be a part of .NET MAUI:

Simple and easy to use
Native playback of media files from remote http(s), embedded and local sources
Native media notifications and remote controls
Queue and playback management by default
Playback status (Playing, Buffering, Loading, Paused, Progress)
Events for media handling to hook into

Depends on

[x] https://github.com/microsoft/microsoft-ui-xaml/issues/5172

vhugogarcia commented 3 years ago

Additionally for livestreaming video support. I hope baseflow can release a MAUI version soon.

However, I also think MAUI could integrate it. The essentials one is pretty basic and does not support live streaming videos.

MhAllan commented 3 years ago

Related to #1125

Askar938 commented 3 years ago

I totally agree with @vhugogarcia, the essentials functionality is very basic and does not really help solving real-world problems. A concrete example is the ability to play audio in the background, it is a common expectation among app users that their audio keeps playing even when the app is in the background or the screen is locked. If .NET MAUI can support standard use-cases like this, I am sure that will make life easier for a lot of developers.

davidbuckleyni commented 3 years ago

U do no of the media element ? It was brought in xammy forms i dont know if it got out of experimental though

davidbuckleyni commented 3 years ago

Additionally for livestreaming video support. I hope baseflow can release a MAUI version soon.

However, I also think MAUI could integrate it. The essentials one is pretty basic and does not support live streaming videos.

Cameraview will support the stream but we cant expect micrsoft to do all the heavy coding on this.

davidbuckleyni commented 3 years ago

But thats the whole point if u want streaming or video calling etc u use third party stuff.

Media Element is not part of essentialals and is in the main forms stuff.

I totally agree with @vhugogarcia, the essentials functionality is very basic and does not really help solving real-world problems. A concrete example is the ability to play audio in the background, it is a common expectation among app users that their audio keeps playing even when the app is in the background or the screen is locked.

If .NET MAUI can support standard use-cases like this, I am sure that will make life easier for a lot of developers.

Askar938 commented 3 years ago

U do no of the media element ? It was brought in xammy forms i dont know if it got out of experimental though

I do and it doesn't support the basic scenario I mentioned above of playing audio in the background as far as I know. Watch this video by Gerald Versluis , he managed to play audio in the background with MediaElement on iOS, but he had to write some platform-specific code in order to make it work. When it comes to Android, I have yet to find someone who has made it work with MediaElement.

ghost commented 2 years ago

We've moved this issue to the Future milestone. This means that it is not going to be worked on for the coming release. We will reassess the issue following the current release and consider this item at that time.

niksedk commented 2 years ago

If we could get the native window handle for a control, it would be a good start - see https://github.com/dotnet/maui/issues/6843 It is fairly easy to implement libmpv/libvlc etc.

emceelovin commented 2 years ago

I created a scoped service that uses the partial class pattern which only builds certain code on certain platforms in the Platform/ folder tree. there is a SetView(View view) partial method that Views supply themselves to, and depending on the platform, calls MediaPlayer.Hwnd/NSObject. LibVLC already has specific Views for Android, UWP, WPF, WinForms, etc, but there is no MAUI View, so i'm creating a ContentView, which obviously derives from a View, and using that combined with its Rect information since MAUI composes everything to one window. Which is fine, but you have to constrain VLC's rendering based on the size of that View in relation to where it's contained. At least this is my goal. I'm sure i'll have more work and this wont be as easy as it sounds.

emceelovin commented 2 years ago

In the case of Windows, assuming the HWND returned by WinRT.Interop.WindowNative.GetWindowHandle() is where all of the views are composed to, that should be good enough for MediaPlayer.HWnd if MediaPlayer allows specifying constraint Rect(the IView's width/height etc.) possibly.

It looks like VLC only supports scaling of a video, which is fine, but we would have to figure out the scale by taking the View's Rect information in relation to the parent HWND's RECT information and calculate a scale factor, but if that even works, VLC probably will render to the HWND's top left corner. We would have to figure out a way to move it, right?

What we need is a PR or fork of LibVLC and LibVLCSharp that has an IView property in MediaPlayer which will make it support MAUIs virtual view style composition. That way it can use the information from IView for width/height and what not.

There's also VideoView that UWP, WPF, Uno, etc. build on for displaying video it seems. Maybe we can just create a custom VideoView implementation based on IVideoView VideoView docs

MAUI supports multiple windows, but I'm not sure if theyre new W'S_OVERLAPPEDWINDOW's or WS_CHILD's on Win32. If there's the ability to make them a child of a Window in MAUI, then we could create a whole new child window and use that as the VLC render target and have perfect coordinates etc.

UPDATE After visiting the direct3d docs since it's been quite some time, it doesn't seem like its possible to solve the issue with a single parent native window. Creating a swap chain against an HWND on Direct3D will clear the whole HWND. I haven't messed with Direct3D since D3D9 came out, so maybe D3D11 allows restricting rendering to certain area's in an HWND and not clearing the whole HWND before presenting a new frame. As mentioned above, and I'm also asking on the VLC's own repo for direct3d advice and trying to help them, if we had the ability to create child Window's in MAUI and not just new top level windows, we could solve this problem instantly.

This is all a shot from the hip, so anyone else can chime in!

emceelovin commented 2 years ago

FWIW, according to the LibVLC conversation I'm having on their internal github, there is a LibVLCWinUI in the works. There's already LibVLCAndroid, LibVLCiOS, and LibVLCMac. those could all be used respestively in Platform/Android,iOS,MacCatalyst, and Windows with the partial class platform specific pattern.

chenlung commented 2 years ago

I find VLC to be a bit unstable on Mac. What about mpv or something else?

emceelovin commented 2 years ago

I find VLC to be a bit unstable on Mac. What about mpv or something else?

Oh yeah? I haven't used it on MacOS. The goal right now was to solidify the interfacing and classes to achieve cross platform audio video. An abstraction of some kind that sews all the platforms together in the Platforms folder. A for now solution of you will, so I'm totally ok with whatever will get to that point right now, and then if there's a better way down the road, move to that implementation. Just having something at this point will be great. I'll have to checkout mpv. Anything is fine with me as long as we can find something that'll allow us to use it on windows. All other platforms have media playing abilities I believe. The problem with anything OS specific in cross platform libraries, is not only do you need an implementation per, you need an implementation to sew it all together, but per platform building code and partial classes really comes in clutch for that.

emceelovin commented 2 years ago

I was successfully able to play media in my test program, but no matter what View you use with WinRT.Interop.WindowNative.GetWindowHandle(), all return the same HWND :(. I'm not sure of the actual internals of MAUI, so I'm not sure if it's even possible to have an HWND per control, or how it's done internally with the handlers. When rendering media with LibVLCSharp.Shared.MediaPlayer, it renders directly to the main applications window as assumed. Super close, but this won't work. It's very performant also. Runs great, but the execution isn't desired.

Maybe if there really are child HWND's, we could EnumChildWindows() on the main window HWND and find the actual view we want to render the media output to? Another shot from the hip. Maybe someone can chime in on the actual structure of MAUI and how it maps exactly composes the UI tree.

emceelovin commented 2 years ago

@niksedk

If we could get the native window handle for a control, it would be a good start - see #6843 It is fairly easy to implement libmpv/libvlc etc.

Exactly. Everything would be perfect and the problem would be solved basically forever.

johnthepro commented 2 years ago

I found an abandoned repo that might make for a good start to migrate over. Maybe sometime much smarter then I can look it over.

https://github.com/adamfisher/Xamarin.Forms.VideoPlayer

fkahhaleh commented 2 years ago

I was successfully able to play media in my test program, but no matter what View you use with WinRT.Interop.WindowNative.GetWindowHandle(), all return the same HWND :(. I'm not sure of the actual internals of MAUI, so I'm not sure if it's even possible to have an HWND per control, or how it's done internally with the handlers.

@emceelovin Could you please link the working sample you have? looking to just playback notifications for now in .NET MAUI. thanks!