search

m1k1o / neko

A self hosted virtual browser that runs in docker and uses WebRTC.
https://neko.m1k1o.net/
Apache License 2.0
5.96k stars 449 forks source link

Adjusting gstreamer Video Quality? #225

Closed RossKellyZA closed 1 year ago

RossKellyZA commented 1 year ago

Question: How do you create a custom gstreamer pipeline or set one of the pre installed defaults to change the video quality settings? I have most probably missed something silly, but I have followed the documentation and recommendations from other "issues" I cannot seem to work it out. If I try using the built in gstreamer pipelines (base, good, bad, ugly), then the video stream just displays black. I have also tried manually setting NEKO_VIDEO_CODEC and NEKO_VIDEO_BITRATE but they seem to be ignored when viewing through chrome://webrtc-internals/.

Essentially I want to try and find the minimum bitrate/resolution/framerate combination I can use for the content I am streaming. This is for a few reasons. Lower the hardware required to host the containers, minimize the bandwidth requirement for the people receiving the stream etc

Setup/Environment

What I have tried ***I have tried all the pipeline presets even though I just gave one example below

  1. Editing /base/DockerFile. image
  2. Editing .env file and adding NEKO_VIDEO=gstreamer1.0-plugins-bad
  3. Adding -e "NEKO_VIDEO=gstreamer1.0-plugins-bad" variables directly into the ./start-server script.
  4. Hosting the latest images on Azure and editing the docker-compose.yaml file.

Whenever I try and set NEKO_VIDEO, the video stream is returned as a black screen. And Ideas?

I have also logged into the neko UI as admin and change the resolution from the dropdown. This does work, but I would imagine this is just using one of the default pipelines and the bitrate still seems high per resolution/framerate. I would like to play with the bitrate more and see the differences with a custom pipeline.

I feel like I am most probably just doing something stupid, but any point in the right direction would be appreciated. Also a big shout out to all the contributors. This is a great project and my plan is to contribute back once I have my head around everything.

m1k1o commented 1 year ago

NEKO_VIDEO needs to be your custom gstreamer pipeline, e.g.:

ximagesrc display-name=:99.0 show-pointer=true use-damage=false ! video/x-raw,framerate=25/1 ! videoconvert ! queue ! vp8enc target-bitrate=2000000 cpu-used=4 end-usage=cbr threads=4 deadline=1 undershoot=95 keyframe-max-dist=25 min-quantizer=4 max-quantizer=20 buffer-initial-size=6500 buffer-optimal-size=7500 buffer-size=8500

There you can tweak all settings and different encoding pipelines.

The same for NEKO_AUDIO:

pulsesrc device=auto_null.monitor ! audio/x-raw,channels=2 ! audioconvert ! opusenc inband-fec=true bitrate=128000

When setting those environment variables (don't need to be set in Dockerfile, they can be just added when running the container, see how configuration can be passed) other variables that control bitrate/fps are not being used, since you specify it in your custom pipeline.

EDIT:

On the docs page are listed installed packages, that contain various collection of GStreamer plug-ins and elements that can be used. E.G. h264 is in Package – GStreamer Ugly Plug-ins. Meaning, we have those packages installed and you can use all plugins that they provide.

They are as listed:

RossKellyZA commented 1 year ago

Thanks for explaining @m1k1o That has cleared things up. I was viewing plugins as presets and trying to set them that way.

I have also moved all my environment variables to my docker-compose.yaml file now which is making things much easier. I am fairly new to the world of docker so just coming up to speed.

When I use the NEKO_SCREEN, NEKO_MAX_FPS, NEKO_VIDEO_CODEC, NEKO_VIDEO_CODEC these all seem to be working now as well as setting up a pipeline which overrides most of the settings as you mentioned

Am I correct in say that you still need to set the NEKO_VIDEO_CODEC manually because this does not seem to change when setting it in the pipeline automatically? Also when using VP9 when changing the video bitrate it does not seem to affect anything, but when using h264 it does work. I will continue working with h264.

Thanks again

m1k1o commented 1 year ago

 Am I correct in say that you still need to set the NEKO_VIDEO_CODEC manually

Yes, you need to set that one. But its only needed for WebRTC so it can announce correct video codec used. You specify it then in your pipeline additionally.

RossKellyZA commented 1 year ago

With some clarification everything is working as expected