search

Chocobozzz / PeerTube

ActivityPub-federated video streaming platform using P2P directly in your web browser
https://joinpeertube.org/
GNU Affero General Public License v3.0
12.74k stars 1.46k forks source link

Support H265/HEVC for Livestreams/VODs #6330

Open MattyBoombalatty opened 2 months ago

MattyBoombalatty commented 2 months ago

Describe the problem to be solved

After spending the good part of today editing a plugin I forked to enable H264 NVENC encoding, I realized that H265 only seems to be supported in VODs, not Livestreams. I'm not sure if this is an issue on my end, or with the browser (I tried several). The replay transcodes fine, and is viewable as normal

Describe the solution you would like

Enable H265/HEVC live video playback in browser.

Chocobozzz commented 2 months ago

I realized that H265 only seems to be supported in VODs, not Livestreams.

Can you describe the problem? Is the player unable to play the live stream? Or is the server unable to transcode it?

MattyBoombalatty commented 2 months ago

I'm able to transcode it fine - I have a plugin and it works pretty well. However, the browser itself will not play the video during a livestream, only during the replay.

  1. Livestream with HEVC profile
  2. Video shows as live, but there is no video on screen
  3. Terminate livestream
  4. Replay video, which plays back in browser correctly

The only issue is that the video does not play back in browser. Please note that HEVC profile is enabled for VODs as well. Here's a link to my plugin if you wanted a better idea of how it works, in case I missed something.

https://gitea.nicecrew.digital/matty/peertube-plugin-nctv-nvenc-transcode

MattyBoombalatty commented 2 months ago

Here's the error - I was incorrect, this does happen with VODs too, on some browsers but I think it depends on what format they were uploaded in. Not sure.

Failed to play video
The video failed to play due to technical issues.

HLS.js error: mediaError - fatal: true - manifestIncompatibleCodecsError
\"reason\":\"no level with compatible codecs found in manifest\"

You can see a test video here.

It seems to work on mobile browsers (at least on Android) - perhaps it's an issue with desktop browsers not supporting H265? Either way, I figured I'd ask.

MattyBoombalatty commented 2 months ago

Update:

H265 encoded VODs will play on Windows and Android mobile devices across Chromium based browsers (Brave, Chrome, Edge) Live videos will not. Perhaps it's the FFMPEG options that are passed in? Who knows.

H265 encoded VODs do not play on Linux browsers (tested Firefox, Chrome and Brave). Unfortunately I believe this is simply a limitation of supported devices.

It would still be nice to have Live videos encoded in H265 play on supported browsers and hardware, so I'll leave this open in an attempt to assist in future proofing PeerTube.

Chocobozzz commented 2 months ago

It would still be nice to have Live videos encoded in H265 play on supported browsers and hardware, so I'll leave this open in an attempt to assist in future proofing PeerTube.

Can you paste the web browser error when you try to play a H265 live?

MattyBoombalatty commented 2 months ago

There is no error. Audio plays, but not video. It's just a black screen.

emansom commented 2 months ago

I'd recommend switching to VP9 instead, broader compatibility and should be supported by NVENC as well.

Edit: doesn't seem to be implemented in ffmpeg. Weird. AV1 a possibility, or card doesn't support it?

If AV1 is not supported and your goal is to increase video quality, with bandwidth being less of a concern: try H264 at 12Mbps bitrate per YouTube's recommendations.

MattyBoombalatty commented 2 months ago

We already do use H264 NVENC and it works fine. I was just hoping to be able to support hevc_nvenc for future proofing and bandwidth constraints.

emansom commented 2 months ago

We already do use H264 NVENC and it works fine. I was just hoping to be able to support hevc_nvenc for future proofing and bandwidth constraints.

HEVC/H265 is dead due to licensing issues. Use AV1 for future proofing and bandwidth saving. 😉

MattyBoombalatty commented 2 months ago

Well unfortunately the GPU in my server cares less about licensing issues and more about encoder support, of which, AV1 is not supported. ;)

emansom commented 2 months ago

Well unfortunately the GPU in my server cares less about licensing issues and more about encoder support, of which, AV1 is not supported. ;)

It's an industry wide concern, this is why web browser compatibility of H265/HEVC isn't broad.

AV1 has a permissible license, allowing web browsers and media appliances to implement software fallbacks without fear of legal ramifications.

AV1 implements similar in goal features to H265/HEVC, which results in similar increase in quality and bandwidth saving.

If you have the available budget: the Intel Arc A310 is a very good choice as hardware accelerated encoder. It supports H264, VP9, H265/HEVC and AV1.

MattyBoombalatty commented 2 months ago

I am unfamiliar with Intel GPUs as they're relatively new to the market. I can budget for it, but that would also involve ensuring that there are Linux drives for it. I have a Tesla P4 at the moment and have been relatively pleased with it. I could, in theory, purchase one for research and see where it goes. I appreciate the heads up.

emansom commented 2 months ago

I am unfamiliar with Intel GPUs as they're relatively new to the market. I can budget for it, but that would also involve ensuring that there are Linux drives for it. I have a Tesla P4 at the moment and have been relatively pleased with it. I could, in theory, purchase one for research and see where it goes. I appreciate the heads up.

The kernel, Mesa drivers, VAAPI and ffmpeg support are all there in the latest stable releases. On rolling-release distro's like Fedora Rawhide/CentOS Stream 9 (+ ELRepo and locally recompiled Mesa to re-enable proprietary codecs) or Arch Linux it'd work.

There is the problem of fan control however, this is currently not yet implemented.

https://youtu.be/zjGieJci3xE

MattyBoombalatty commented 2 months ago

Much appreciated.

Chocobozzz commented 2 months ago

@MattyBoombalatty Can you provide a permanent live stream link + a vod video link so I can try to find what's the cause of your issue?

MattyBoombalatty commented 2 months ago

@Chocobozzz sure, when would be a good time for you? Did you want me to set up an HEVC encoded livestream and give you the link while it's going?

Chocobozzz commented 2 months ago

Did you want me to set up an HEVC encoded livestream and give you the link while it's going?

Yes please

MattyBoombalatty commented 2 months ago

Okay, I'll do that tonight. Our time differences make this more difficult :smile:

MattyBoombalatty commented 2 months ago

Are you available for me to do this now?

Chocobozzz commented 2 months ago

Yes

MattyBoombalatty commented 2 months ago

Here is the HEVC livestream:

https://nicecrew.tv/w/tpC3YaHrwTTn8Ys3gkJE8w

and an HEVC VOD

https://nicecrew.tv/w/mLbNbdTbhFFviFsD8zmtfp

Chocobozzz commented 2 months ago

Thanks, unfortunately the issue seems more complex that I thought. Can you try AV1 and tell me if it works?

MattyBoombalatty commented 2 months ago

The transcoding GPU on my server does not support AV1, unfortunately. In the future I will likely upgrade GPUs, but I'll see if I can put together a plugin for it. Thanks for your help!

SimplyCorbett commented 2 months ago

I don’t know about live-streaming but HEVC VOD worked fine for me with the transcoding debug plugin. I think I had to modify a line or two in the source code as well.

Linux web browsers work but have to be compiled with hevc support. For windows you need to download the hevc codec in the windows store and use edge.

im not sure if this has changed since hevc has gained more mainline support however with the web browsers as this was over a year ago.