Use FLOSS formats for video transcoding

[Sp3r4z]

Hello, I've seen Peertube transcode video after upload (really good idea, for security, standardization…) but, here I am: Peertube use MP4 & AAC, the first is under patent encumbed the second is a proprietary format… I think it's not really interesting and satisfying for a FLOSS project promoting proprietary formats.

I think something like: VP9+Opus in a WebM container, is really more appropriate for this project.

Let me know.

PS: I know that x264 and MP4 are more CPU-friendly since low-level CPU instruction sets exist for them ; but for me a FLOSS project should promote free (as speech) formats regardless.

[Serkan-devel]

How can mkv-files be played on the web? It seems to be similar to webm

[Sp3r4z]

@Serkan-devel what do you mean? Is your comment linked to the purpose of the issue?

[Serkan-devel]

mkv is a FLOSS format, that I can only run on VLC but not on the web

[Sp3r4z]

Right, I get that. But why talking about .mkv? I propose WebM container with VP9+Opus inside, because it's readable on Internet browsers (like Firefox), which is free (as speech and as beer).

[rigelk]

@Serkan-devel Matroska is just a container standard, equivalent to WebM but unfit for web diffusion.

The encoding time for VP9 is still 5-fold of H.264, and the upcoming AV1 is gonna be 20-fold (I know it's yet to be optimised but it probably won't be any lower than VP9). So there's that.

We could let administrators choose the encoding they want and leave the actual behaviour as default.

[Sp3r4z]

@rigelk You're right, for the transcoding task but, here is my point: it requires less storage resources and bandwidth. I know, it's a balance between CPU and storage/bandwith, but for videos and streaming, I think it's more important than CPU usage.

As I said on this #479 in mediainfo.txt transcoding seems to pose some problems (30fps but the original was 1fps, worse sound quality… ). Maybe just improve transcoding parameters, can be something really "in-between".

[Chocobozzz]

We use H264/AAC because we use the videostream package to stream data inside a video element. These codecs are the only ones compatible with all web browsers. It's currently not the case for VP9.

[rigelk]

@Chocobozzz Apple is pushing for H264/AAC, so it's unlikely they will support VP9 anytime soon. It doesn't mean we should abide by their vision. But I have to admit searching for a WebM muxer doesn't yield a lot of results…

[Sp3r4z]

@Chocobozzz You're right, I've thought about that too. My idea was just "open the discussion" and see if it can be a good idea for future. But yeah, Safari (desktop and ios), doesn't want to heard about that :/

@rigelk ffmpeg, doesn't do the job you search? (I'm not a video expert, so maybe I ask for something obvious :s )

[abakkk]

Are there any instances that don't transcode in mp4 ? I saw old webm video in peertube and it works fine.

[zez3]

In light of the latest browser updates I would also be tempted to have a look at the new AV1 codec https://en.wikipedia.org/wiki/AV1 ff, chrome already implemented some WebRTC experimental versions. https://www.phoronix.com/scan.php?page=news_item&px=GStreamer-1.14.0-Released

[Sp3r4z]

@abakkk Some instances don't transcode, mp4 trancsode is an option which can be disable quite simply.

@zez3 it's actually experimental, for sure it'll be, in the future, a good thing to thought about (really, I think that too). Unfortunately, right now it's not really a good idea for mass usage. But we'll see, I guess.

[ghost]

I also don't install proprietary software on my GNU/Linux PC which renders PeerTube instances unusable. Most of the time I check things out on PeerTube from my smartphone.

It's not an easy decision to be made here, I understand that, but hopefully this will solve out somehow.

[Sp3r4z]

@Zig-03 "I also don't install proprietary software on mu GNU/Linux" well ok, but which message your message refers to? Can you explain a bit more?

My original message (the first one, in fact) is totally about that, so maybe yours is link to that one?

[ghost]

Hey Sp3r4z, I'm new to github (well the account is aged but I've never been active on github) so I don't yet know how to reply to certain message, but yes, I was referring to your initial post. :)

[Sp3r4z]

@Zig-03 Right ;)

It's exactly the purpose of this issue (and my first message), because I can't accept and think about a "free project" which promote no-floss formats… especially when server does a transcoding from VP9+Opus to MP4+AAC…

[Chocobozzz]

We just use a format that can be played on evergreen web browsers. @Sp3r4z your vp9 videos would not work on iOS and Safari. Our goal is to decentralize video streaming, but how could we convince people to upload their video on PeerTube if 5-10% of the users just cannot play the video?

Of course we'll switch and promote an open format, but only when it will be supported by these web browsers. I don't think this format will be VP8/VP9 because of Apple, and Firefox has issues with webm videos that they don't seem to be in a hurry to fix. IMHO this open format will be AV1, but it's very young so I don't think we'll be able to add it to PeerTube before many years.

[rigelk]

We could transcode additional formats, so long as we keep MP4+AAC. But then that's adding to the transcoding cost, and we still have to write a muxer so it's a long task for very little impact at the moment.

[Sp3r4z]

@Chocobozzz I know all of that, and it's totally and clearly explain in this discussion. I fully understand why my proposition isn't as good as I wanted, no problem. For AV1, it's not "apple compliant" (right now) so problem is the same here… Hope it'll implemented in Safari soon. I also know about MP4 cpu encoding performance, as I said.

For Firefox, I've no idea about but Youtube do streaming on webM and I've no problem with it. Maybe an encoding problem :/

And it's not really interesting, for videos, the @rigelk having two formats of video…

I don't know the MP4+AAC licence conditions, but we should look at it really carefully and precisely!

[Serkan-devel]

Google punish apple for not implementing webm, by NOT displaying YouTube videos in 4k in the safari browser?

[rigelk]

@Sp3r4z maybe https://video.stackexchange.com/a/14699 could answer your question.

if you're charging viewers for h.264 / MPEG-4 AVC content you do need to pay license fees. Even though x264 / ffmpeg are Free with a big F, they are just software libraries for encoding video streams into the H.264/MPEG-4 AVC format, which is covered by the MPEG patent.

[Sp3r4z]

@rigelk Right, no problem for this point, then (for MP4) For AAC, it's here: http://www.via-corp.com/us/en/licensing/aac/licensefees.html

So maybe we can close this issue, because MP4+AAC is the best format, because of Safari.

[Serkan-devel]

But not everyone uses safari

[Sp3r4z]

@Serkan-devel Look at this: Wikipedia browser stats Something like 13% min for Safari in general, and 17,6% for mobile… as @Chocobozzz said: it's not reasonable, actually, cutting video access for Safari User (especially on iOS).

[rigelk]

The browser incompatiblity might be a non-issue with https://github.com/brion/ogv.js (which videojs has a plugin for). It's basically a javascript decoder for Ogg Vorbis/Opus/Theora/WebM. Performance-wise of course it's below a native implementation, but the demo yields good enough results (the results have significantly improved since 2015, but that's just for the xiph.org reference ;)).

[abakkk]

I understand apple users are numerous but this is very difficult to find good packages to play mp4 with Firefox in Fedora for example.

[tcitworld]

@abakkk There's dozens of answered questions and tutorials on how to configure Firefox for h264 support out there.

EDIT : first one found https://ask.fedoraproject.org/en/question/95637/h264-videos-not-working-in-firefox/

[Serkan-devel]

Maybe if there is a webassembly implementation of those open source codecs as @rigelk described it, one can give iOS/Safari users the same as their open source counterparts while not relying on patented ones on an open source project

[rigelk]

I've been playing around with ogv.js (on a friend's Safari, ofc) and it does solve the problem seamlessly.

I've also done a PR to https://github.com/hartman/videojs-ogvjs so that we can integrate a fresh version with our videojs player.

Note that it only gives us room to ignore incompatibilities browsers might have. It doesn't solve the main issue: videostream/media-render lack seeking support for WebM and the like.

[Serkan-devel]

fair enough

[ghost]

It depends how big the performance hit is, but ogv.js would avoid having to deal with battles between companies trying to use codecs to monopolize platforms. If the CPU use increases by 10% that's feasible, if it's 100% then perhaps not.

In my own use case, standardizing upon mp4 just means that I won't be able to view most PeerTube videos, which defeats the object if the goal is to have a free software alternative to YouTube.

[rigelk]

@bashrc it's difficult to say. The latest benchmark done by the ogv.js developper dates back from 2015. At that time iPhone 4s, iPod Touch 5th-gen and iPad 3 had trouble with anything above 240p ; iPhone 5C-5S-6-6Plus with anything above 460p.

Now, since 2015 the library has made leaps forward in terms of performance apparently, so I wouldn't rely on these numbers. Note the README performance section hasn't been updated either, but the changelog since 2015 shows some notable perf increase (~15-25% for the video).

[ghost]

@rigelk

Note that it only gives us room to ignore incompatibilities browsers might have. It doesn't solve the main issue: videostream/media-render lack seeking support for WebM and the like.

by videostream/media-render you mean live streaming? Aren't there any available, FOSS options for this?

[rigelk]

@Zig-03 no, I mean the libraries we use in https://github.com/Chocobozzz/PeerTube/blob/develop/client/src/assets/player/video-renderer.ts as refered to by Chocobozzz's comment.

EDIT: btw, there is #151 talking about live streaming :wink:

[zez3]

Why not simply include the option for use of the AV1 library in config like AV1transcoding. Ideally with a warning that it may currently impact your cpu when re-encoding. e.g. https://ffmpeg.org/general.html#Alliance-for-Open-Media-libaom

For streaming AV1, at the moment, I think we need some gpu style processor help or some cloud encoding service like this guys used: https://bitmovin.com/bitmovin-supports-av1-encoding-vod-live-joins-alliance-open-media/ "a Lenovo T540p notebook with an i7-4800MQ, 8GB RAM running Ubuntu 14.04. It would take 8 hours and 42 minutes to encode a 1080p@24fps 40 second long sequence (Tears of Steel Teaser) with a target bitrate of 1.5Mbps."

[rigelk]

Existing implementations both in encoding and decoding will continue to evolve in unconsistent manner until the AV1 spec is frozen. Until then we cannot consider AV1 support, or otherwise things could break anytime. But things should settle fast so I'm confindent we'll be able to reconsider soon :slightly_smiling_face:

[zez3]

@Chocobozzz it seems that somewhere in January: " Apple only recently joined AOMedia as a member and in doing so added significant weight to the initiative.

The iPhone-maker had previously backed HEVC or H.265, but as CNET notes, using the software in products can be difficult and requires negotiating royalties with three different HEVC patent licensing groups. " So i guess now we'll wait and see when the next MacOS release brings

[rezonant]

I think if WebM/VP8/9 were to become a potential transcoding target format, then some instances will opt to not transcode into mp4/h264, so using ogv.js as a fallback on Safari is a must.

Hopefully with AV1 none of that will be needed.

[AndreaMonzini]

Personally i will wait AV1 ( long wait) or optional VP9+Opus ( i hope shorter wait ) before use PeerTube. For my personal video projects i am evaluating ogv.js or Safari® users can download the file and open it with VP9 player like VLC.

[Jorropo]

@Sp3r4z I know this issue is old but on #679 @rigelk made a rework of video transcoding and i'm working for use VP9+opus on webm in transcoding, the choice will be be made admin :

• mp4 only
• webm only
• mixed (webm for <= 360 ppx res, mp4 > 360) (cause cpu is optimized for mp4 but if you look in 360 the cpu optimization's isn't useful and webm take significantly less space) (probably set to default choice)

[rigelk]

@Jorropo the problem is that instances caring less about codecs will federate with those doing 2 & 3. Their users won't understand why videos just don't work on their devices.

[utack]

mixed (webm for <= 360 ppx res, mp4 > 360)

Seems like that should be turned around

• You barely save any data when an already small video is 20% smaller
• Modern devices where people want to stream HD will have the capabilities to decode VP9
• VP9 saves more data at high resolution that it does at low resolution, that is what it was designed for

[AndreaMonzini]

@rigelk i would test and use as personal choice only webm (vp9 + Opus), is it possible to use the @Jorropo solution?

Maybe i would add requirements in the description or in the video too.

Thank you.

[Jorropo]

@AndreaMonzini not yet sorry, wait beta-11

[AndreaMonzini]

@Jorropo ok thank you ! Where i can check the beta-11 availability?

[Jorropo]

@rigelk Mixed will require some work, cause I need to change video broadcast but webm already work. peertube.jorropo.ovh/videos/watch/8f62627e-4b2b-4e7f-b449-cf3e601764e1, this video is webm (thx pgAdmin)

[rigelk]

@Jorropo the webm muxer doesn't support seeking, so your video plays but the seekbar doesn't work for instance. That and the end user and federating instances need a way to filter out these videos. Honestly this is more work than you might think :worried:

[Jorropo]

@rigelk hum, ok, that a problem of data size (cause we do approximately frameCount*sizeOfOneFrame and we binary seek to that (and that doesn't work with webm cause sizeOfOneFrame is for mp4), not impossible)

[Jorropo]

@AndreaMonzini I don't really know :wink: in fact we are at prerelease for 10 so I think not to long (in less than 1 week if you compile yourself the node)

[rigelk]

@Jorropo fixing that problem is tied to fixing it in jhiesey/videostream, as we rely on it to render the media (see https://github.com/Chocobozzz/PeerTube/blob/develop/client/src/assets/player/video-renderer.ts ). That means writing a muxer akin to https://github.com/jhiesey/videostream/blob/master/mp4-remuxer.js for WebM.