Add support for passing Textures to the capture feed

[brandonjmatthews]

This is a great package! I wanted to modify it for a slightly different purpose and thought it would be worth submitting a PR of my changes so others can find and use them. Currently the package is limited to passing one (or more) Unity camera feeds through to virtual webcams. This PR adds support for individual textures to be passed through the virtual webcam one by one. Reasons for this may include passing a video texture frame by frame or static image through the virtual webcam.

A few notes about my approach:

• I opted to require the user to pass a Texture2D in every time they want to update the frame. I initially considered simply passing it a render texture however I felt this approach was cleaner. It is a more versatile interface for a developer to be able to control the rate at which frames are passed and allow them to pass single Textures into the feed.
• A render texture still needs to be blitted to as using Texture2D.GetNativeTexturePtr() showed black when I tested it.
• I considered adding the changes to the existing UnityCapture script, however given that script requires a camera I decided to separate it out. This means a bit of repeated code however I have minimised this where possible, reusing enums from the existing UnityCapture script.
• I have also included a simple demo script and scene showing how it might be used.

[schellingb]

Hi there Thanks for the pull request! I guess that is a use case worth supporting (-:

Maybe a base class with two sub-classes would be nicer for reducing code reuse? I do want to keep the number of files someone has to import into their project at a minimum though. I might put three classes in one file as long as the sub-classes are minimal.

Btw. the capture filter (the receiving part on the other end) has some logic in it to stop showing the incoming image if no new frame arrives for 1000 milliseconds. Having a manual call to CaptureSendTexture might lead to this happening unexpected. Maybe a flag needs to be passed along that this capture is not guaranteed to be at a decent frame rate. I assume your use-case is for something with more than one frame per second?

[brandonjmatthews]

I was thinking along the same lines about base and sub classes, however I didn't want to drop a refactor PR on you with no prior warning.

Yes my use case is >1 fps so I didn't come across that in my testing as I was only testing one update per frame.

An alternative is just putting the updates in a constant cycle (LateUpdate/similar) and hold that Texture2D. Then whenever necessary that Texture2Dcan be updated by a function call. That could tie in reasonably well with an inheritance structure as the base class could handle when to update the capture filter and the sub classes could just handle what to update it with (camera frame/provided frame).

[schellingb]

Would you be fine with using it like this?

public class CaptureRenderTexture : MonoBehaviour
{
    public int width = 320, height = 240;
    Texture2D RenderTexture;
    UnityCapture.Interface CaptureInterface;
    int y = 0;
    Color color = Color.red;

    void Start()
    {
        // Create texture and capture interface
        RenderTexture = new Texture2D(width, height, TextureFormat.ARGB32, false);
        CaptureInterface = new UnityCapture.Interface(UnityCapture.ECaptureDevice.CaptureDevice1);
    }

    void OnDestroy()
    {
        CaptureInterface.Close();
    }

    void Update()
    {
        // Draw next line on texture 
        for (int x = 0; x < width; x++) RenderTexture.SetPixel(x, y, color);
        y += 1;
        if (y > height) { y = 0; color = new Color(color.g, color.b, color.r); }

        // Update the capture texture
        RenderTexture.Apply();
        CaptureInterface.SendTexture(RenderTexture);
    }
}

Meaning not having a separate UnityCaptureTexture MonoBehaviour but a public low-level capture Interface class. The only reason to have CaptureRenderTexture as a MonoBehaviour was to have Unity call the OnDestroy function. But because this approach needs a custom script that calls SendTexture anyway I think it's not too much to ask. Actually it runs fine without manually calling Close() because Unity runs the garbage collector on end which calls the finalizer of the class.

Btw. if you set mipmap = false when constructing the Texture2D the extra blit copy into a RenderTexture becomes unnecessary. Maybe this can be detected and a warning can be thrown. Edit: Or we just accept textures with mipmaps and ignore everything but the main texture when capturing. That seems to be easier.

[brandonjmatthews]

Looks good to me! I'll split out the capture instance specific parts into a small internal class (not MonoBehaviour), abstracting away the DllImports etc. into an Interface class as you have suggested.

Quick questions:

• Should CaptureInterface.SendTexture() also require the other options (DoubleBuffering, ResizeMode etc.) or should they be passed through the constructor of the Interface?
• Should CaptureInterface.SendTexture() return the ECaptureSendResult, Log the errors as in the switch statement in UnityCapture.cs, or both? (I'm thinking just return the Result)
• And, If any new instance is created with an already in use a ECaptureDevice, is an error thrown by CaptureCreateInstance, or should I add an error and keep track of used ECaptureDevices in some Static list?

[schellingb]

Yeah I was thinking of keeping things super light with SendTexture looking like this: public ECaptureSendResult SendTexture(Texture Source, bool UseDoubleBuffering = false, EResizeMode ResizeMode = EResizeMode.Disabled, EMirrorMode MirrorMode = EMirrorMode.Disabled) These parameters are fine to be changed at any time so anything else would make things more complex for the main camera capture class.

Multiple instances with the same ECaptureDevice can already happen (and if both send at the same time the receiving application receives a flickering stream of both sources) but it does not lead to any problems or even crashes so I think we can leave as is. I guess it would be nice to catch it inside the DLL plugin but for now it's ok as it is.

Once your done, I'll go ahead and add a Timeout parameter. It will specify the number of milliseconds until the capturing can be considered inactive. A Timeout of 0 would then always show the last sent image (until both Unity and the receiving application are stopped).

[brandonjmatthews]

Yep, that's what I have done. I left the Enums in UnityCapture.cs since they would still be accessible and it keeps it cleaner (as opposed to using UnityCapture.Instance.Enum.EnumValue everywhere). Let me know if there are any further things you want changed or any issues you find.

[schellingb]

Yeah leaving them in UnityCapture is desired so things don't break for people who update from the old version. Looks good to me! I'll go ahead and merge it. Thanks so much for the quick response :-)

[brandonjmatthews]

Awesome, no problem at all! Thanks for the suggestions and allowing me to contribute something!

[schellingb]

Committed the timeout parameter which makes this complete. Thank you for contributing!