search

pikvm / ustreamer

µStreamer - Lightweight and fast MJPEG-HTTP streamer
https://pikvm.org
GNU General Public License v3.0
1.68k stars 226 forks source link

Elgato CamLink support (NV12 format and so on) #97

Open karepiu opened 3 years ago

karepiu commented 3 years ago

I got below with Elgato Cam Link. Any idea ? 


[2021-03-01 09:21:19 kvmd.service] --- kvmd.apps.kvmd.streamer           INFO --- Console: ================================================================================
[2021-03-01 09:21:19 kvmd.service] --- kvmd.apps.kvmd.streamer           INFO --- Console: -- INFO  [87.886    stream] -- Device fd=8 opened
[2021-03-01 09:21:19 kvmd.service] --- kvmd.apps.kvmd.streamer           INFO --- Console: -- INFO  [87.886    stream] -- Using input channel: 0
[2021-03-01 09:21:19 kvmd.service] --- kvmd.apps.kvmd.streamer           INFO --- Console: -- ERROR [87.966    stream] -- Requested resolution=1920x1080 is unavailable
[2021-03-01 09:21:19 kvmd.service] --- kvmd.apps.kvmd.streamer           INFO --- Console: -- INFO  [88.046    stream] -- Using resolution: 3840x2160
[2021-03-01 09:21:19 kvmd.service] --- kvmd.apps.kvmd.streamer           INFO --- Console: -- ERROR [88.046    stream] -- Could not obtain the requested pixelformat=JPEG; driver gave us unsupported
[2021-03-01 09:21:19 kvmd.service] --- kvmd.apps.kvmd.streamer           INFO --- Console: -- ERROR [88.046    stream] -- Unsupported pixelformat=NV12 (fourcc)
[2021-03-01 09:21:19 kvmd.service] --- kvmd.apps.kvmd.streamer           INFO --- Console: -- INFO  [88.046    stream] -- Device fd=8 closed
[2021-03-01 09:21:19 kvmd.service] --- kvmd.apps.kvmd.streamer           INFO --- Console: -- INFO  [88.046    stream] -- Sleeping 1 seconds before new stream init ...
[2021-03-01 09:21:20 kvmd.service] --- kvmd.apps.kvmd.streamer           INFO --- Console: ================================================================================
``
mdevaev commented 3 years ago

NV12 format is not supported. Use UYVY, YUYV, RGB24, JPEG or MJPEG.

karepiu commented 3 years ago

Is there a way to change it from UI ?

mdevaev commented 3 years ago

No. First you need to make sure that the camera supports the desired formats. Show v4l2-ctl --list-formats-ext -d /dev/kvmd-video.

karepiu commented 3 years ago

`[kvmd-webterm@pikvm ~]$ v4l2-ctl --list-formats-ext -d /dev/kvmd-video ioctl: VIDIOC_ENUM_FMT Type: Video Capture

    [0]: 'NV12' (Y/CbCr 4:2:0)
            Size: Discrete 3840x2160
                    Interval: Discrete 0.033s (29.970 fps)
    [1]: 'NV12' (Y/CbCr 4:2:0)
            Size: Discrete 3840x2160
                    Interval: Discrete 0.033s (29.970 fps)
    [2]: 'YU12' (Planar YUV 4:2:0)
            Size: Discrete 3840x2160
                    Interval: Discrete 0.033s (29.970 fps)

[kvmd-webterm@pikvm ~]$ `

mdevaev commented 3 years ago

Sorry, but no chance to enable it. It has no supported formats.

karepiu commented 3 years ago

That is interesting. Seems the question was correct - Elgato is not supported

Elgato seems to be power house currently for all streaming capture devices ( half of the world uses it for OBS ) - strange that so popular and unfortunately expensive hardware does not work. Bummer

Nevertheless - I would suggest a feature - dropdown allowing to change the format in UI.

mdevaev commented 3 years ago

In fact, I think I could add support for this thing, but it requires separate research. The reason why this is not supported is very simple: I don't have the money to buy and test such devices. Especially since this is not a good fit for Pi-KVM. This thing does not know how to compress an image on its own and compressing video on a raspberry 4k will require huge hardware resources.

mdevaev commented 3 years ago

The format is configured once and then used repeatedly. If ustreamer supported one of the formats of this device, it would work without explicitly specifying the format on your part.

karepiu commented 3 years ago
  1. Makes sense that it does not do any compression - it is usually connected to powerful streaming PC. The question would be - is it worth to support it, even though it is not good for this kvm, just because it is popular - hard to say.
  2. What if device supports more then one supported format ? Wouldn't allowing ability to choose from supported format be beneficial ? ( imho having control is always better than not having one )
mdevaev commented 3 years ago
  1. This device will not be able to run on a RPi as efficiently as on a PC simply because of the lack of resources on the first one.
  2. The video format is a low-level setting that can be done via the config. If you look at /etc/kvmd/main.yaml, then you will see that the mjpeg format is configured there by default. This setting is sufficient for absolutely all cases of using the recommended usb hdmi grabber. Similar for hdmi csi.

Let's imagine that ustreamer supports all three formats of your. What format will you choose? NV12? And how will it differ from the other two? This is a technical setting that does not make sense to change the user at all. It doesn't affect absolutely anything, it's just the format of transmitting pixel data from the device to the streamer.

For example, CSI HDMI supports UYVY and RGB24. By default, the first one is used. The second one is never needed at all, it gives nothing and is even worse supported by the driver. Once you need to choose the best format for the device and never change it. You needed this setup because your device didn't work. If all three formats were supported, the driver would choose one of them and just use it.

karepiu commented 3 years ago

I understand it and it is ultimately your decision and your product - so whatever you decide stands :).

My experience though suggest that there is lots of edge cases everywhere and hiding option from user just because something happens automatically ( or as I sometimes say "automagically" ) is almost never good decision. What if I have device that supports UYVY and RGB24 but for some reason implementation of UYVY is broken. Most of power users probably would override main.yaml etc. but would have to figure it out first that this is an issue. Having dropdown would allow anybody to try different setups and choose the best for their situation - it may not work in my case as my device simply does not support needed formats but it may be great option for somebody else. Ofc it will not matter for your v3 hardware but for all DIY may be absolutely great option.

Whatever you decide - good luck and Thank you for the great work.

I will just purchase different capture card and try it out again.

mdevaev commented 3 years ago

If UYVY is broken, then you will know it from the log. In Pi-KVM, there are fifty parameters that somehow affect the video. If you follow this logic, you need to ensure that they are configured from the GUI, without giving preference only to the format. You are experiencing a problem because you are using a device that is not on the recommended list. In addition, as far as I can tell, changing the format alone would not be enough. It seems that other settings need to be changed, such as the list of screen resolutions.

mdevaev commented 3 years ago

uStreamer-specific task. It will be done in the future.

Vincent056 commented 10 months ago

I have a patch added NV24 and NV16 supported needed for rk3588 https://github.com/Vincent056/ustreamer/commit/793f904c3f3ab2f2d06fcd7549f7dee1f3776482

also this is needed, it is tested on orangePi5 Plus: https://github.com/Vincent056/ustreamer/commit/6a1c16f5334764c8238f97ec13c17bceb7aefe67

Eric-168 commented 10 months ago

I have a patch added NV24 and NV16 supported needed for rk3588 Vincent056@793f904

also this is needed, it is tested on orangePi5 Plus: Vincent056@6a1c16f

@Vincent056 First of all, thank you very much, I am using your rk3588-b branch The process is this: git clone -b rk3588-b https://github.com/Vincent056/ustreamer.git cd ustreamer make And replace the successfully compiled ustreamer and ustreamer-dump to /usr/bin chmod +x ustreamer chmod +x ustreamer-dump I use ustreamer --device=/dev/video0 --host=0.0.0.0 it no longer prompts me Video capture is not supported by device Moreover, I can also see the video stream on HTTP port 8080 I don't know what I did wrong but I still can't see the video stream in pikvm

Eric-168 commented 10 months ago

1 2 3

Eric-168 commented 10 months ago

My previous version was uStreamer 5.37. I don’t know if it’s because the new uStreamer 5.43 is incompatible with my pikvm.

Eric-168 commented 10 months ago

4 I modified it by /etc/kvmd/main.yaml inside --device=/dev/video0 Now I can get the video stream in pikvm, but I don't know why its color is yellowish. Is it because of the capture card firmware? Thank you very much again. @Vincent056 @mdevaev

Vincent056 commented 10 months ago

4 I modified it by /etc/kvmd/main.yaml inside --device=/dev/video0 Now I can get the video stream in pikvm, but I don't know why its color is yellowish. Is it because of the capture card firmware? Thank you very much again. @Vincent056 @mdevaev

could you post what you have in the end of journalctl -u kvmd it might be a issue with where you placed ustreamer bin file I suspect

Eric-168 commented 10 months ago

4 I modified it by /etc/kvmd/main.yaml inside --device=/dev/video0 Now I can get the video stream in pikvm, but I don't know why its color is yellowish. Is it because of the capture card firmware? Thank you very much again. @Vincent056 @mdevaev

could you post what you have in the end of it might be a issue with where you placed ustreamer bin file I suspectjournalctl -u kvmd

I was wrong. Regarding the color issue, it was because the captured machine turned on the night mode.

pl804 commented 1 week ago

Hi @Eric-168 @Vincent056

I also have an extra Elgato CamLink lying around that I'm trying to get working with PiKVM, to avoid buying another component.

My supported formats are similar to @Eric-168 (although slightly different):

root@pikvm bin]# v4l2-ctl --list-formats-ext
ioctl: VIDIOC_ENUM_FMT
        Type: Video Capture

        [0]: 'NV12' (Y/UV 4:2:0)
                Size: Discrete 3840x2160
                        Interval: Discrete 0.033s (30.000 fps)
        [1]: 'NV12' (Y/UV 4:2:0)
                Size: Discrete 3840x2160
                        Interval: Discrete 0.033s (30.000 fps)
        [2]: 'YU12' (Planar YUV 4:2:0)
                Size: Discrete 3840x2160
                        Interval: Discrete 0.033s (30.000 fps)

Like @Eric-168, I cloned @Vincent056's branch and ran the exact same commands, and edited main.yaml accordingly to also specify video0.

The good news is I no longer get the unsupported device error.

The bad news is I hit an error during capture. I also no longer get resolution options in the dropdown.

I noticed @Eric-168 somehow was able to set the resolution to 1920x1080. For some reason, mine won't set to anything other than 4K, even after setting the resolution on the target host to 1024x768. I was worried 4K was producing output that was too large for PiKVM to handle.

Any idea what I'm doing wrong?

Thanks.

-- INFO  [602.756      http] -- Starting HTTP eventloop ...
-- INFO  [603.010    stream] -- Device fd=8 opened
-- INFO  [603.010    stream] -- Using input channel: 0
-- INFO  [603.010    stream] -- Probing device format=YUYV, stride=3840, resolution=1920x1080 ...
-- INFO  [603.011    stream] -- Querying device format ...
-- INFO  [603.011    stream] -- Query pix format: width=3840, height=2160, pixelformat=NV12, field=1, bytesperline=3840
-- INFO  [603.011    stream] -- Current device set to for pix: width=1920, height=1080, pixelformat=NV12, field=0, bytesperline=3840
-- INFO  [603.011    stream] -- Supported device formats:
-- INFO  [603.011    stream] --   NV12
-- INFO  [603.011    stream] --   NV12
-- INFO  [603.011    stream] --   unsupported
-- ERROR [603.096    stream] -- Requested resolution=1920x1080 is unavailable
-- INFO  [603.096    stream] -- Probing device format=NV12, stride=7680, resolution=3840x2160 ...
-- INFO  [603.096    stream] -- Querying device format ...
-- INFO  [603.097    stream] -- Query pix format: width=3840, height=2160, pixelformat=NV12, field=1, bytesperline=3840
-- INFO  [603.097    stream] -- Current device set to for pix: width=3840, height=2160, pixelformat=NV12, field=0, bytesperline=7680
-- INFO  [603.097    stream] -- Supported device formats:
-- INFO  [603.097    stream] --   NV12
-- INFO  [603.097    stream] --   NV12
-- INFO  [603.097    stream] --   unsupported
-- INFO  [603.178    stream] -- Using resolution: 3840x2160, size:24883200
-- INFO  [603.178    stream] -- Using format: NV12
-- INFO  [603.324    stream] -- Using HW FPS: 0 -> 30 (coerced)
-- INFO  [603.324    stream] -- Using IO method: MMAP
-- INFO  [603.408    stream] -- Requested 5 device buffers, got 5
-- INFO  [603.439    stream] -- Capturing started
-- INFO  [603.439    stream] -- Using JPEG quality: 80%
-- INFO  [603.439    stream] -- Creating pool JPEG with 4 workers ...
-- INFO  [603.440    stream] -- Capturing ...
-- ERROR [604.441    stream] -- Mainloop select() error: Invalid argument
-- INFO  [604.441    stream] -- Destroying workers pool JPEG ...
-- INFO  [604.443    stream] -- Capturing stopped
-- INFO  [604.459    stream] -- Device fd=8 closed