Window capture

[gregersn]

Is it possible to capture a specific window? And not the whole screen?

I want to work with a video that is playing with audio, and want that to always be playing, and kept in sync, but importing the video in as a source does not seem to play audio, so the thought was to be able to capture it in.

[brunoherbelin]

I see your point, and agree that we could do something to help in the situation you describe (which could very well also help more people).

Several approaches: 1- capture video stream from another program: under OSX, it could be with syphon 2- gstreamer command-line: it is possible to create a pipeline to read a video file and separate the video from the audio streams, sending them to two distinct targets. One could thus read a video, send the audio to the speaker and stream the video to vimix (e.g. network stream on localhost). 3- allow one video inside vimix to play audio.

I don't like 1- because it is platform-specific I like 2- because it is really leveraging on the power of GStreamer: i'll investigate. You might prefer 3-, which could be a user-friendly solution (i'll check).

[brunoherbelin]

Here is an example of gstreamer pipeline to split audio and video reading a media file (solution 2):

gst-launch-1.0 uridecodebin uri=file:///home/username/video.mp4 name=dec dec. ! queue ! videoconvert ! autovideosink dec. ! queue ! audioconvert ! autoaudiosink

NB: replace home/username/video.mp4 with your filename

Simple UDP stream..

gst-launch-1.0 uridecodebin uri=file:///home/username/video.mp4 ! queue ! videoconvert ! jpegenc ! rtpjpegpay ! udpsink port=5000 host=127.0.0.1

...with its simple UDP receiver

gst-launch-1.0 udpsrc port=5000 ! application/x-rtp,encoding-name=JPEG,payload=26 ! rtpjpegdepay ! jpegdec ! autovideosink

Combined : play audio locally and stream video:

gst-launch-1.0 uridecodebin uri=file:///home/username/video.mp4 name=dec dec. ! queue ! videoconvert ! jpegenc ! rtpjpegpay ! udpsink port=5000 host=127.0.0.1 dec. ! queue ! audioconvert ! autoaudiosink

What's next would be that I allow to create a network receiver in vimix. I'll check how to do this!

[RKelln]

I was playing around with gstreamer and vimix on Ubuntu 22.04 today and thought I'd share some results, it worked really well!

You can use v4l2loopback to make a virtual webcam of a window and then add that in vimix as a Connected source.

 sudo modprobe v4l2loopback video_nr="42" 'card_label=virtcam' exclusive_caps=1 max_buffers=2

 gst-launch-1.0 ximagesrc xid=0x3200004 use-damage=0 show-pointer=0 ! videoconvert ! queue ! v4l2sink device=/dev/video42

Where xid can be gotten through xwininfo.

I also tried doing a window from another machine on the LAN:

server (computer with window you want to see in vimix):

gst-launch-1.0 ximagesrc xid=0x6800002 use-damage=0 show-pointer=0 ! queue ! videoconvert ! x264enc tune=zerolatency ! rtph264pay ! udpsink host=192.168.X.X

(where 0x6800002 is replaced by your window id and 192.168.X.X is replaced with the client ip address)

client (where vimix is running):

gst-launch-1.0 udpsrc ! application/x-rtp,media=video,clock-rate=90000,encoding-name=H264,payload=96 ! rtph264depay ! avdec_h264 ! videoconvert ! queue ! video/x-raw,format=YUY2 ! v4l2sink device=/dev/video42

There is likely some better ways of doing this, but thought someone might be interested. I couldn't get the jpeg encoding version of this working at all. :|

[brunoherbelin]

Hey ! Thanks for sharing ! I'll think of a place to gather such user experience.

I take the opportunity to add the ref to shmdata that also can help in this matter 👍 https://github.com/nicobou/shmdata

[RKelln]

First time I'm working with gstreamer so those aren't expert advice just experiments that eventually worked out. I did take a stab at shmdata too which looks really cool but the install did not go smoothly (there are no built packages for 22.04 in that ppa). The instructions from https://gitlab.com/sat-mtl/tools/shmdata did work however. But I had issues with it working consistently.

How do you use shmdata sink in vimix I only saw the virtual webcam and SRT, or is that just a performance option in the gstreamer pipeline?

[brunoherbelin]

Its been a while since this request and I propose to close the issue as two new features are now available in vimix 0.8.2 to answer to the expressed needs.

• audio support for media sources (minimal and experimental, but answering to the need)
• screen capture selection of individual window