Unity Video Capture backend

[graphemecluster]

(Discussion: https://github.com/letmaik/pyvirtualcam/discussions/51)

Many of the files are rewrote for this pull request. Just as you said, I'm not including the install scripts though the shared.inl is necessary. And I'm finally not going to touch the UnityCaptureFilter.cpp file.

Looking at the source code of UnityCaptureFilter.cpp:

• Yes, the conversion of RGBA to BGRA inside the file is unavoidable without modify it.
• A maximum number of 74 devices can be installed.
• The devices are named name #index except for index 0, which is named name. Yes, the default value of name is Unity Video Capture, but it can be changed by the user by the install script argument UnityCaptureName. The number of devices installed is specified by the argument UnityCaptureDevices with the default value 1. All devices have different CLSID in the format 5C2CD55C-92AD-4999-8666-912BD3E700XX.

With the above information:

• We parse the device parameter by the following condition:
  • If device is missing, we iterate through 0 to 74 to check for the first device registered;
  • If device is a number-like string from 0 to 74, the index is used;
  • Otherwise the device is assumed to be the device name; we iterate through 0 to 74 to check if device matches the name given by the register key; or an error is fired if there are no matches.
• The device() function returns the device name given by the register key, not necessary Unity Video Capture.

A few more things to notice:

• I think it won't works on a notable number of Windows devices due to the RegGetValueW method.
• The red-blue channels are not switched at least for my device in the final display.
• I am not changing argb to bgra in image_formats.h due to consistency, but of course you can if you want.
• All uint32_t are changed to int32_t, though only the argb_to_argb method has actual use to negative numbers.
• I used std::wstring instead of std::string.
• The fps parameter is not used currently.
• I don't know if we also need to include the shared.inl file in setup.py.
• I think it is completed but I'm not sure.

Please let me know if there's anything unclear or forgotten.

[letmaik]

Looks great! Do you mind if I commit to your branch and fix up the remaining issues?

[graphemecluster]

By all means! I'm trying to fix it too. Edit: Although the errors are fixed, be sure to check if it is completed.

[graphemecluster]

If the RegGetValueW method does not work at all, try replacing

if (RegGetValueW(HKEY_CLASSES_ROOT, guid, L"", RRF_RT_REG_SZ, NULL, &str, &size) != ERROR_SUCCESS) return false;

with the following:

HKEY key;
if (RegOpenKeyExW(HKEY_CLASSES_ROOT, guid, 0, KEY_READ, &key) != ERROR_SUCCESS) return false;
if (RegQueryValueExW(key, L"", NULL, NULL, (LPBYTE)str, &size) != ERROR_SUCCESS) return false;

I don’t know which one is better if both work though.

[letmaik]

I did the following changes:

• For now, let's not use unicode. This has to be done across all backends, and in theory it makes sense, but should be a separate PR.
• I renamed some bits to be more consistent. I think I'll rename the folder as well, to native_windows_unity_capture.
• SendIsReady, as you had it earlier, unfortunately has to go into send() as it will always be false if no app is capturing the cam yet.

I'll do some more testing etc. later today, also it has to be added to CI. The color is still wrong for me. Can you try in multiple apps? The latency.py sample gives you a good idea of the expected color (see terminal output).

• Some minor fixes and cleanup.

[graphemecluster]

Note that std::stoi() return 3 for strings like 3DCamera, which is not expected.

[letmaik]

Note that std::stoi() return 3 for strings like 3DCamera, which is not expected.

You're right, didn't think of that. It may make sense to remove the numeric case altogether for now since it becomes ambiguous if multiple backends do the same. It probably makes more sense for each backend to expose all available devices and then let the user choose, based on some criteria, but that's out of scope in this PR.

[graphemecluster]

OK, let’s remove it then. Oh, I didn’t edit the README either.

[letmaik]

I've added the proper pixel format conversions now and it looks fine for me. Could you give it a try? If it doesn't look right for you, then your frames are probably not in RGBA channel order. This would be fully opaque red: [255, 0, 0, 255]

[letmaik]

I just stumbled upon these: https://github.com/obsproject/obs-studio/discussions/4476 https://stackoverflow.com/q/66854423 Once we're done here those two should get a follow-up response.

[letmaik]

I updated the README and added a new sample which sends transparent gif frames to the cam. I included some instructions on how to configure OBS to capture in RGBA. From my side I'm done now. Would be good if you could give it a final test before I merge it. Note that you can also download the wheels from the artifacts here: https://github.com/letmaik/pyvirtualcam/suites/2528611452/artifacts/54812494

[graphemecluster]

Sorry for letting you wait. I am in a different time zone from you. Currently there are some problems to my computer so I am not able to test it. It should work on most computers without my test. Thank you for your patience.

[letmaik]

I found an issue when using multiple cameras which requires a fix upstream https://github.com/schellingb/UnityCapture/issues/17.

[graphemecluster]

For now, let's not use unicode. This has to be done across all backends, and in theory it makes sense, but should be a separate PR.

So should all of them be changed to wide-characters now? This is a minor problem though.

[letmaik]

For now, let's not use unicode. This has to be done across all backends, and in theory it makes sense, but should be a separate PR.

So should all of them be changed to wide-characters now? This is a minor problem though.

Sure, I won't have time for it though. So far, all the virtual cameras didn't need it as the names are all ascii. And I have a feeling that probably no one is using custom names for unity capture. Do you have a use case?

[graphemecluster]

For now, let's not use unicode. This has to be done across all backends, and in theory it makes sense, but should be a separate PR.

So should all of them be changed to wide-characters now? This is a minor problem though.

Sure, I won't have time for it though. So far, all the virtual cameras didn't need it as the names are all ascii. And I have a feeling that probably no one is using custom names for unity capture. Do you have a use case?

I think so too. Actually I don't need it either, so let's just put it aside.