USB capture Cards

[magdesign]

Trying to use Elgato Cam-Link 4k capture card to feed directly into IQsynth.

First tested the input with "Cheese" webcam tool, there I am able to select the internal webcam or the Cam Link and see the preview.

Then installed the uv4l driver: sudo apt-get install v4l-utils v4l2loopback-utils v4l2loopback-source checked with: v4l2-ctl --list-devices

Cam Link 4K: Cam Link 4K (usb-0000:00:14.0-3):
    /dev/video2
    /dev/video3

Chicony USB2.0 Camera: Chicony  (usb-0000:00:14.0-8):
    /dev/video0
    /dev/video1

to see that it finds the device.

After starting IQSynth I am only able to select the internal webcam as video source:

but under "Audio-IN" I see that it sees the audio device of Cam Link:

Here the video in report:

since in OBS-Studio I am able to see it as an uv4l capture source there is the question if something is missing for IQSynth?

Any hints welcome, in the meantime I try to make a loopback device with ffmpeg and/or test firmware flashing: https://github.com/tolga9009/elgato-gchd

[magdesign]

Tested the Cam Link 4 k with: https://webcamtests.com/ it works in the browser.

Then created a loopback device with ffmpeg:

sudo modprobe v4l2loopback devices=1 exclusive_caps=1

v4l2-ctl --list-devices # Get the video devices for the camlink and dummy.

ELGATO=/dev/video2

V4LOOP=/dev/video4

ffmpeg -f v4l2 -input_format yuyv422 -framerate 60 -video_size 1920x1080 -i $ELGATO -pix_fmt yuyv422 -codec copy -f v4l2 $V4LOOP

which creates a "Dummy video device" and this works with the webcam testsite as well, but still nothing visible in IQSynth

[ghost]

looks like it hasn't got any read buffers from the screen shot, as I mention in the docs I need a capture device which can supply at least 2 buffers, but I need to see a debug log please to confirm this is the current problem

[magdesign]

ok. I plugged in the Cam Link stick, started IQ Synth in verbose mode (./IQSynth 1) , went to the configuration, video in report and closed the application:

here the log: https://gist.github.com/magdesign/acb5ea9c8d8c0134a9907b7914e38208

[ghost]

I seem to only see streaming options in that capture device setup. At the moment, the capture process requires a device & driver setup which have the following abilities: Allows Memory Mapping Capture Formats: 24bit RGB|BGR|YUYV And at least 2 ReadBuffers

This has worked with the devices I have at hand, but I'll likely need to look into streaming only devices at some point.

[ghost]

I'm going to review this bit of code next, to see if there is any low hanging fruit I can pick off.

[ghost]

this looks broken, prev working devices no longer generate valid images, hmmm...

[magdesign]

sounds awful.... can I help you somehow? (are you coding in C?)

[jimruxton]

I am trying to do something similar. I am using ffmpeg to create a videoloopback device from a YouTube Live stream. IQSynth also doesn't seem to pick this up on /dev/video2 . Here is the command I am using ffmpeg -re -i $(youtube-dl -g -f 95 https://www.youtube.com/watch?v=9Auq9mYxFEE) -f v4l2 /dev/video2 -f alsa default

The device shows up as a dummy video device

$ v4l2-ctl --list-devices
Dummy video device (0x0000) (platform:v4l2loopback-000):
    /dev/video2

I have been using it in the program Pure Data and the video shows up there. I wanted to try out IQSynth as it looks awesome.

[ghost]

@magdesign the thing I'm currently chasing is a shader/setup bug, not sure what/when it changed but...the end shader seems to no longer be receiving the correct texture cords

Hi @jimruxton, I've never run a video loopback before, that looks like fun. I'll have a go soon, but I suspect there are a few reasons this wont work with IQSynth at the moment, completely guessing without trying it first... A: its a stream only (I cant handle those just yet) B: might be in a compressed format (also cant handle) but I may well be very wrong, thanks for the pointers on those commands, I'll def come back to this

[ghost]

Hi, I've reverted the shaders to older versions (not using VBOs) all seems to be working again for now. Released new version, wont help @jimruxton, but it might solve something for @magdesign. I didn't notice until now that in the picture above the device is available in IQSynth so it might just be that you weren't seeing a picture (due to the bug I may have just fixed)

[jimruxton]

Thanks for the info. @johnday-github . Maybe I will see if I can convert the stream to make it look more like a usb device using []https://github.com/wlhe/uvc-gadget . Looking forward to iQSynth being able to take streams :)

[ghost]

the youtube-dl project is really cool, will come back to that. fyi: for now, I got a setup with 2 USB cams but it will only run 1 at a time (this is with a very old cheap webcam) will chase this and start looking at streaming only architecture. Not promising anywhere quickly, but would like to get this working.

[ghost]

ah the 2 usb cams at same time is an OLD v4l2 known problem, not my fault! best ref so far: https://superuser.com/questions/431759/using-multiple-usb-webcams-in-linux

[jimruxton]

Not sure if this is helpful information for you but I'll pass it along. When I use the ffmpeg command below the dummy videoloopback device ( /dev/video2) now shows up under externals. When I go to turn it on I see one frame of the stream. After that if I go to turn it off or close the program everything freezes and the only way to get out is reboot the machine. I wouldn't expect it to work as you said it didn't support streaming. It does however seem encouraging that the loopback device appears under externals and one frame shows up :) Command is below . Thanks for giving this a whirl. ffmpeg -re -i $(youtube-dl -g -f 95 https://www.youtube.com/watch?v=9Auq9mYxFEE) -pix_fmt yuyv422 -f v4l2 /dev/video2 -f alsa default

[ghost]

cool, will try to reproduce here

[ghost]

whoop,whoop! got to same stage... freeze on a frame, but just seen a 1000 frames duplicated msg from ffmpeg, so might be that end, still good start! note: to anyone else reading this, you need to install the v4l2loopback kernel module from https://github.com/umlaeute/v4l2loopback to get going, and the youtubeDL tool from https://github.com/ytdl-org/youtube-dl#installation

[jimruxton]

Great, let me know if you want me to do any more testing on my end.

[ghost]

OK, I've not played with these tools before, a lot more learning needed, I think I need to get a working feed first. Even a 'basic' feed from youtube-dl to vlc player does the same freeze on first frame. Have you got a working command line?

[ghost]

although I can get pre-recorded to show in vlc but no live streams yet

[ghost]

k, got pre-recorded to show up in IQSynth but still no luck with live streams in any way. I need to check if the change I made needs to be conditional or is generally OK.

[ghost]

OK, have released a new version 0.17.2 I don't think this solves too much, but the changes are:

• memory mapped video devices where it gets stuck during dequeuing buffer, will pass if the buffer is full
• and can now toggle capture device during dequeuing buffers

I haven't got a live feed to run at all, but a pre-recorded feed seems to work, once you have a v4l2loopback installed and running, and youtube-dl install, I successfully used the following command to start a stream (must do this BEFORE starting up IQSynth)

ffmpeg -re -i $(youtube-dl -g -f "[height <=? 720]" https://www.youtube.com/watch?v=VNGFep6rncY) -pix_fmt rgb24 -f v4l2 /dev/video0 -f alsa default

note: change video0 to whatever your loopback device is prob a few other options will work as well... @jimruxton let me know if it works for you

btw: @magdesign were you able to get any picture from webcam yet?

[jimruxton]

@johnday-github I am able to see the streaming video not freezing in Pure Data using the command below. ( Same as previous )

ffmpeg -re -i $(youtube-dl -g -f 95 https://www.youtube.com/watch?v=9Auq9mYxFEE) -pix_fmt yuyv422 -f v4l2 /dev/video2 -f alsa default

It also shows up in the ffplay window. ie.

ffplay /dev/video2

I have also been using the command

ffmpeg -re -i $(youtube-dl -g -f 96 https://www.youtube.com/watch?v=9Auq9mYxFEE) -f v4l2 /dev/video2 -f alsa default

which works however I don't see the loopback device under externals in your software with this command. Wondering if you have tried different stream resolutions , replacing 96 with the other quality numbers below ` 91 mp4 256x144 HLS 197k , avc1.42c00b, 30.0fps, mp4a.40.5@ 48k

92 mp4 426x240 HLS 338k , avc1.4d4015, 30.0fps, mp4a.40.5@ 48k

93 mp4 640x360 HLS 829k , avc1.4d401e, 30.0fps, mp4a.40.2@128k

94 mp4 854x480 HLS 1380k , avc1.4d401f, 30.0fps, mp4a.40.2@128k

95 mp4 1280x720 HLS 2593k , avc1.4d401f, 30.0fps, mp4a.40.2@256k

96 mp4 1920x1080 HLS 4715k , avc1.640028, 30.0fps, mp4a.40.2@256k (best)`

I haven't tried your latest software yet. FYI I am on Ubuntu 19.10 $ uname -a Linux jim-Lenovo-Y720 5.3.0-53-generic #47-Ubuntu SMP Thu May 7 12:18:16 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

This last command also works well for me streaming to VLC however yes the first command is freezing in VLC for me too??

[jimruxton]

@johnday-github . I downloaded your new software and using the ffmpeg command below (same as before) IT WORKS. Stream looks good , doesn't freeze and plugins seem to work though I need to spend some time learning how to use your software. Congrats! I hope it will work for you too.

` ffmpeg -re -i $(youtube-dl -g -f 95 https://www.youtube.com/watch?v=9Auq9mYxFEE) -pix_fmt yuyv422 -f v4l2 /dev/video2 -f alsa default

`

[ghost]

:) Cool, live feeds still freeze for me, I wonder if the difference is your logged into youtube?

[jimruxton]

Hmm. If I do a clean reboot without going to a browser and starting YouTube it seems to work the same. Not sure how I would be logged into YouTube, though who knows if Google owns it they may have a way :) . Does the stream freeze when using ffplay after starting the ffmpeg command? By the way my version of ffmpeg is

ffmpeg version 4.1.4-1build2 Copyright (c) 2000-2019 the FFmpeg developers built with gcc 9 (Ubuntu 9.2.1-4ubuntu1)

youtube-dl version

2019.11.05

[ghost]

ah, yes quite a few differences in versions, I'll update my ffmpeg soon. Got a bit side tracked with this ticket. @magdesign Not sure if it can be closed yet, what you think?

[magdesign]

Sorry I had no time to test this one, using the grabber card now to feed the mapping software with the output of IQSynth... Up to you, I can imagine that in the next few month there will come a better loopback driver...

[ghost]

think I'll close this one then, any new bug can start a new ticket