is just for streaming ?

[OurIslam]

ffmpeg -i mp3 -i mp4 c copy outmp4 rendering it has this / it can do this ?

[lukasf]

This is a decoding library, so there is currently no encode/transcode functionality.

[OurIslam]

but all this stuf exist in uwp media transcode what it has in interopx

[lukasf]

Well, technically you could use FFmpegInteropX for decoding, and pass the MediaStreamSource to the UWP transcoding APIs. Some people have done that. But you are limited in the encoding/transcoding target formats to what UWP has to offer. You cannot use FFmpeg stuff for that.

[OurIslam]

it can not decode / deserilize the cluster list from binary which ffmpeg can ,

[lukasf]

I am not sure I understand your scenario. If you want to do transcoding, you could use MediaTranscoder:

https://learn.microsoft.com/en-us/uwp/api/windows.media.transcoding.mediatranscoder.preparemediastreamsourcetranscodeasync?view=winrt-22621

var ffmpegSource = await FFmpegMediaSource.Create...
var mediaSource = ffmpegSource.GetMediaStreamSource();
MediaTranscoder.PrepareMediaStreamSourceTranscodeAsync(mediaSource, outputStream, encodingProfile);

What do you mean with cluster list?

[OurIslam]

clusterlist mean the audio or video file have track position hex address list

[OurIslam]

each address contains of data[] byte

[OurIslam]

var ffmpegSource = await FFmpegMediaSource.Create... var mediaSource = ffmpegSource.GetMediaStreamSource(); MediaTranscoder.PrepareMediaStreamSourceTranscodeAsync(mediaSource, outputStream, encodingProfile);

i tested it , its good , but i want something new in uwp , here everything uwp have in https://learn.microsoft.com/en-us/windows/uwp/audio-video-camera/transcode-media-files

if possible please try to give the Matroska.Muxer

[lukasf]

No, as I said you are limited to UWP transcoding APIs and formats, no Matroska.

FFmpegInteropX only has the FFmpeg decoding part. There are currently no plans to add encoding.

[brabebhin]

Perhaps we should consider transcoding. This seems to be in popular demand.

[OurIslam]

and its been long years passed no one success in this part , many people tried

[lukasf]

It sure is popular. But it means loads of work. It is not only about designing a new good and flexible API surface (decoding, encoding, trancoding, probably also custom raw data feeding) and implementing all the FFmpeg interop stuff. For encoding, you also need to add many new libs. At least x264, x265, av1 and probably a bunch more, to cover the popular formats. There are not many encoders included in core FFmpeg. Then people will want GPU accelerated encoding, so you have to add NVEnc, QuickSync, AMD whatever they call it, because there is no commin API for video encoding. Adding new libs is always a pain, because the MSVC toolchain is often not properly supported by the lib's build system.

It is especially the lib part that makes me not want to do this. Writing code can be fun, especially if you create something new from scratch. But the fun factor of fighting with build systems is more like zero for me.

Additionally, encoding is incredibly complex. If you take a look at popular encoders such as Handbrake, there are loads of config options for every single codec, and every codec has totally different configuration. You cannot simplify this down to a "quality" slider. And if you did, some people would not be happy about the outcome, because every file is different and would need different settings for best quality / file size offtrade. So you'd probably need both, some kind of simplified config scheme for people who "just want to encode", but also an advanced config scheme for all the codec specific settings.

Personally, I don't have any use for video encoding in my projects. So it is loads of (at least partly unpleasant) work for something I don't need at all. If someone would properly pay me for this, I'd probably do it, but I guess that's not the case ^^

I would support this if you want to do all the heavy lifting @brabebhin, but I do not see myself putting a lot of work into this. It's just too much work. In Germany we call this a "barrel without a bottom" ^^ I could be wrong though, sometimes things are easier than what they seem. But I am not very keen on trying right now.

[brabebhin]

Don't we have av1, x264 and x265 already though?

[lukasf]

We have an av1 decoder, but no encoder. And yes, I have added x264 and x265 encoders a year back or so at an experimental basis. But these are desktop builds currently, and I know I had to do patching at multiple places to get these working. I don't have the patches anymore, and I also don't really remember what the problems were. So there is still some work for them, although they are kind of pre-integrated already. But more libs will sure be required.

[brabebhin]

So for the API part, I'd just support the filter syntax, with input and output stream objects.

As for libraries, yeah, that would be the biggest issue. Linking libraries is always a PITA to deal with. I'd assume most of those we need don't even support uwp builds.

Something like this

https://ffmpeg.org/doxygen/trunk/transcoding_8c_source.html

[lukasf]

This does not even allow you to set the output format. It is only sample code to show how transcoding can be done in general. I don't think that this will help anyone. If you want to provide encoder/transcoder functionality, you must allow configuration of the output format and settings (per stream). And it should be flexible enough to support different use cases. Like having a source stream and a target URI, or source URI and target stream. People have been asking about that already (encode data for sending through rtsp), so if we provide something, it should be flexible enough to support these scenarios. Most probably you will also want to support multiple sources, like raw video stream plus raw audio stream, and encode to one target stream, and you might want to be able to select which of the input streams are encoded and which are dropped.

I agree that a simplified API is also desired, where you just have input stream and output stream, and set video and audio options (being applied to all corresponding streams) and target format options. But if you provide this, people will quickly want more. So I think, if we provide anything in that direction, it should be flexible to cover all the use cases, otherwise we'll have to redo everything quickly.

Setting codec options in FFmpeg happens through AVDictionary, so we could use a generic PropertySet (like we have for FFmpegOptions) per-stream, and additonally a filter string if filtering is required. And then one more PropertySet for target container format options. That way, the configuration would be generic and people just have to use the known and documented options from ffmpeg.

[lukasf]

By the way, I just noticed that x264 and x265 are GPL. So it is basically not possible to use them in a product, unless the product itself is open source. This makes the two most popular encoders unavailable for most apps. AV1 encoder libs are BSD.

[brabebhin]

So I am guessing Microsoft implements its own encoders then. Yeah, the licensing is a show stopper... Unless we can work around it by using the DirectX encoders (haven't researched much in this area). But yeah, a lot of work.

[lukasf]

We can use GPU encoding, but only where the hardware supports it. DX12 encoding is pure HW encoding, so it also does not work where not available. And I would not really go that route, it is pretty new and it currently only supports two formats. I'd rather use HW encoding through FFmpeg. It's just that this requires additional libs.