[AndroidX.Media3.Transformer] EditedMediaItem.Builder missing

[tipa]

Android application type

Android for .NET (net8.0-android)

Description

I am trying to follow Googles "Getting Started" for the Media3 Transformer package to do a media file transformation, documented here.

Steps to Reproduce

In C#, this would be something like the following:

var inputMediaItem = MediaItem.FromUri(uri);
var editedMediaItem = new EditedMediaItem.Builder(inputMediaItem).SetRemoveAudio(true).Build();

The EditedMediaItem class doesn't seem to be present (not bound?), the same seems to be the case for many other classes in the same package/namespace. Maybe the class hasn't been bound (using remove-node) or has only been added in a recent version of Media3, which hasn't been bound yet.

[moljac]

@tipa Thanks a lot for feedback

The EditedMediaItem class doesn't seem to be present (not bound?), the same seems to be the case for many other classes in the same package/namespace.

I will take a look next week. Still on vacation.

Maybe the class hasn't been bound (using remove-node) or has only been added in a recent version of Media3, which hasn't been bound yet.

Everything is on the table. In the meantime I prepared updates for Media3, which did not require too much changes (new dependency packages etc)

[jpobst]

It looks like the EditedMediaItem class did not exist in version 1.0.2, but is now present in the bound 1.1.1 version:

[rgb-drm]

Thanks @moljac for getting this done quickly. I was searching for the tools to add an overlay watermark+timer to video on the weekend and was pleased to come across this conversation already in progress. Good work!

I'm looking to use the Media3.Transformer plus OverlayEffect along with a BitmapOverlay to apply my overlay watermark+timer.

Comparing the 1.1.1 generated OverlayEffect class with the java source code here, there is a constructor required for the OverlayEffect class:

1. The AndroidX.Media3.Effect.OverlayEffect class is included in the latest build but it is currently missing its constructor method (which accepts an input param of [ImmutableList]<TextureOverlay> textureOverlays);

2. The toGlShaderProgrammethod in the OverlayEffect class looks like it needs to call OverlayShaderProgram and not return a SingleFrameGlShaderProgram as it currently does. (Edited this out since SingleFrameGlShaderProgram has been deprecated in a later version of AndroidX/Media so should auto-resolve with a later AndroidX/Media version.)

If there's anything else I can provide to help, happy to oblige. Thanks.

[tipa]

@jpobst the lines I posted in my first post now build with v1.1.1. However, the very next lines of the example are still not working because the setVideoMimeType method is missing

[jpobst]

It looks like setVideoMimeType was added in version 1.2.0, hence it will not be in these bindings.

[moljac]

@jpobst the lines I posted in my first post now build with v1.1.1. However, the very next lines of the example are still not working because the setVideoMimeType method is missing

Keep it open until everything is fixed. Maybe add comment or two what is fixed. Just so we have track of it.

[moljac]

It looks like setVideoMimeType was added in version 1.2.0, hence it will not be in these bindings.

It is in the update queue, but I need to test fixes for MavenNet 1st.

[tipa]

I tested the recently released 1.2.0 and encountered new problems. First there was this compatibility problem that required me to manually reference Xamarin.AndroidX.Collection.Jvm:

Error   NU1107  Version conflict detected for Xamarin.AndroidX.Collection.Jvm. Install/reference Xamarin.AndroidX.Collection.Jvm 1.4.3 directly to project ... to resolve this issue. 
 ... -> Xamarin.AndroidX.Media3.Transformer 1.2.1 -> Xamarin.AndroidX.Media3.ExoPlayer 1.2.1 -> Xamarin.AndroidX.Collection 1.4.3 -> Xamarin.AndroidX.Collection.Jvm (>= 1.4.3) 
 ... -> Xamarin.AndroidX.Preference 1.2.1.9 -> Xamarin.AndroidX.Fragment.Ktx 1.8.2.1 -> Xamarin.AndroidX.Collection.Ktx 1.4.2.1 -> Xamarin.AndroidX.Collection.Jvm (>= 1.4.2.1 && < 1.4.3).

Then I tried again to implement the previously mentioned example in my code:

var inputMediaItem = MediaItem.FromUri(uri);
var editedMediaItem = new EditedMediaItem.Builder(inputMediaItem).SetRemoveAudio(true).Build();
Transformer transformer = new Transformer.Builder(Application.Context)
    .SetVideoMimeType(MimeTypes.VideoH265)
    .AddListener(new Listener())
    .Build();
transformer.Start(editedMediaItem, outputPath);

sealed class Listener : Java.Lang.Object, Transformer.IListener
{
    public override void OnCompleted(Composition composition, ExportResult exportResult) {    }
}

However, I could build it: Error (active) CS0115 'Listener.OnCompleted(Composition, ExportResult)': no suitable method found to override

I think the OnCompleted and OnError interface methods need to be public

[tipa]

I am still unable to use this package with 1.3.1:

• I am now able to implement Transformer.IListener but the interface methods should be made public (they are currently private)
• The AndroidX.Media3.Effect.Presentation class of the bindings inhertis from Java.Lang.Object, while it should be MatrixTransformation - which in turn inherits from Effect, which is the required type for setEffects

var inputMediaItem = MediaItem.FromUri(uri);
var editedMediaItem = new EditedMediaItem.Builder(inputMediaItem)
    .SetEffects(new Effects([], [AndroidX.Media3.Effect.Presentation.CreateForHeight(480)]))
    .Build();
var transformer = new Transformer.Builder(Application.Context)
    .SetVideoMimeType(MimeTypes.VideoH265)
    .Build();
transformer.Start(editedMediaItem, outputPath2);

Cannot implicitly convert type 'AndroidX.Media3.Effect.Presentation' to 'AndroidX.Media3.Common.IEffect'

[tipa]

Same problem with 1.4.1. And I also had to manually reference Xamarin.AndroidX.Collection.Jvm again to install the package along with Xamarin.AndroidX.Preference.

Will there be any efforts in resolving the issues I have reported over the last couple of weeks? I am still unable to use it at all.

[moljac]

Thanks for reporting

Will there be any efforts in resolving the issues I have reported over the last couple of weeks?

Yes. For sure, but I need to see the priorities first. Probably bindings errors/problems 1st.

And I also had to manually reference Xamarin.AndroidX.Collection.Jvm again to install the package along with Xamarin.AndroidX.Preference.

This is not bindings issue but nuget transitive dependencies. We are working on solutions for that.

Thanks

[moljac]

After fixing:

    public sealed partial class Presentation : global::Java.Lang.Object, global::AndroidX.Media3.Effect.IMatrixTransformation

    public partial interface IMatrixTransformation : global::AndroidX.Media3.Effect.IGlMatrixTransformation

    public partial interface IGlMatrixTransformation : global::AndroidX.Media3.Effect.IGlEffect

    public partial interface IGlEffect : global::AndroidX.Media3.Common.IEffect