VideoController cannot be registered with ConferenceController

[leeprobert]

If you are instantiating prefabs with video rendering gameObjects as components then you will need the VideoController to be added programmatically and setup to connect with the ConferenceController and also set the filter for the player name/id. I am using Unity Netcode so the players ID is also the Dolby Conference Player name. This means I need to wait for the Relay to connect and the player prefabs to be created before setting up the VideoController. The code registers itself with the ConferenceController on the Awake method. It also sets up the rendering on the Start method. These both fail as I do not have the ConferenceController injected at the point the prefab is instantiated. There needs to be a public method to defer this setup.

[leeprobert]

My code:

using DolbyIO.Comms.Unity;
using Unity.Netcode;
using UnityEngine;

public class PlayerConferenceConnector : NetworkBehaviour
{
    private VideoController _VideoController;
    private ConferenceController _ConferenceController;
    public override void OnNetworkSpawn()
    {
        Debug.Log($"OwnerClientId : {OwnerClientId.ToString()}");
        base.OnNetworkSpawn();
        GameObject dolbyManager = GameObject.Find("DolbyManager");
        DolbyIOManager _DolbyIOManager = dolbyManager.GetComponent<DolbyIOManager>();
        _ConferenceController = dolbyManager.GetComponent<ConferenceController>();
        GameObject videoScreen = transform.Find("VideoScreen").gameObject;
        _VideoController = videoScreen.AddComponent<VideoController>();
        _VideoController.IsLocal = IsLocalPlayer;
        _VideoController.Conference = _ConferenceController;
        _VideoController.FilterBy = ParticipantFilter.Name;
        _VideoController.Filter = "Player " + OwnerClientId.ToString();

        if(IsLocalPlayer){
            _DolbyIOManager.PlayerName = "Player " + OwnerClientId.ToString();
            _ConferenceController.Join();
            _ConferenceController.StartVideo();
        }
    }
}

As soon as I add the VideoController as a component, it attempts to reference the ConferenceController which doesn't exist yet.

[leeprobert]

I was able to get around this by deactivating the GameObject, setting up the variables, and then reactivating so it called the Awake method. Finally, you need a short delay before calling StartVideo so it has time for the ConferenceController to register the VideoController and wire everything up.

This class is attached to a NetworkObject Prefab for use in a Netcode Relay app.

using DolbyIO.Comms.Unity;
using Unity.Netcode;
using UnityEngine;
using System.Collections;

public class PlayerConferenceConnector : NetworkBehaviour
{
    private VideoController _VideoController;
    private ConferenceController _ConferenceController;
    public override void OnNetworkSpawn()
    {
        Debug.Log($"OwnerClientId : {OwnerClientId.ToString()}");
        base.OnNetworkSpawn();

        GameObject dolbyManager = GameObject.Find("DolbyManager");
        DolbyIOManager _DolbyIOManager = dolbyManager.GetComponent<DolbyIOManager>();
        _ConferenceController = dolbyManager.GetComponent<ConferenceController>();

        // get the child gameObject called 'VideoScreen'
        GameObject videoScreen = transform.Find("VideoScreen").gameObject;
        videoScreen.SetActive(false);
        // add the _VideoController to the 'VideoScreen' gameObject
        _VideoController = videoScreen.AddComponent<VideoController>();

        if(IsLocalPlayer){
            _DolbyIOManager.PlayerName = "Player " + OwnerClientId.ToString();
        }

        _VideoController.IsLocal = IsLocalPlayer;
        _VideoController.Conference = _ConferenceController;
        _VideoController.FilterBy = ParticipantFilter.Name;
        _VideoController.Filter = "Player " + OwnerClientId.ToString();

        videoScreen.SetActive(true);
        StartCoroutine(JoinConferenceAndStartVideo());
    }

    IEnumerator JoinConferenceAndStartVideo()
    {
        yield return new WaitForSeconds(1);
        if(IsLocalPlayer){
            _ConferenceController.Join();
            _ConferenceController.StartVideo();
        }
    }
}

[leeprobert]

I am only seeing video renders on the local player's prefab. Could this be because I am testing locally on one machine and sharing the webcam feed? Does it not allow this?

[leeprobert]

What do I need to do with my non-local VideoControllers? The Dolby.IO PlayerName is the Netcode OwnerClientId and I can see that this is being set as the Filter value, and the FilterType is set to use the name. Is this enough for the ConferenceController to use the right VideoTrack? Should I call Join or StartVideo on the non-local participants? This wouldn't make sense to me. What am I missing here?

[leeprobert]

The other issue I have found is that the PlayerName on the DolbyIOManager can't be set AFTER the Session has opened. So you need to defer opening the session. By doing this:

if(IsLocalPlayer){
            // Get the DolbyManager object and start the session if it is not already started
            GameObject dolbyManager = GameObject.Find("DolbyManager");
            DolbyIOManager _DolbyIOManager = dolbyManager.GetComponent<DolbyIOManager>();
            _DolbyIOManager.OpenSession();
            _ConferenceController.Join();
            _ConferenceController.StartVideo();
        }

I was able to get the remote player video tracks to show! Result. BUT, the client local player track wasn't working. I think this could be an issue with using the same webcam feed for both clients. Will run on two machines to see.

[leeprobert]

Getting close to fixing this. In this screenshot the window on the left is the HOST so was the first client to launch. The window on the right is the CLIENT. As you can see, BOTH video renders are working on the right so it is showing the local and the remote participant. On the left the local participant is showing but NOT the remote.

[leeprobert]

My theory on why the HOST is not working is possibly something to do with the conference alias being set BEFORE the session is opened. The HOST app is the first to create the alias and then when the prefab is created it opens the session on the DolbyIOManager but I wonder if the session alias is not set yet? When other clients are opened and they set the alias on their DolbyIOManager it must see that the alias is a current session and everything is fine.

[djova-dolby]

Hi there, would adding the following public methods essentially be what you are looking for? Since this repo is OSS it would be greatly appreciated if in the future a PR could be provided along with the issue if the fix is known, just to speed things along. Thanks in advance!

[leeprobert]

Thanks for getting back to me. I don't have a lot of experience with building Unity packages or contributing to OS projects but I would be open to giving it a go. There are certainly some other issues I have spotted that could be problematic when trying to use the SDK in this way. One issue is this:

void Start()
        {
            if (_sdk.IsInitialized)
            {
                _sdk.Conference.VideoTrackAdded += HandleVideoTrackAdded;
                _sdk.Conference.VideoTrackRemoved += HandleVideoTrackRemoved;
                _sdk.Conference.ParticipantUpdated += HandleParticipantUpdated;
            }

            if (AutoJoin)
            {
                Join();   
            }
        }

I think this could be the issue I am seeing with the HOST. The Conference alias is set by the host so it is possible that the SDK is NOT initialised and therefore the delegate methods are not being set in some cases.

I am not 100% sure what is happening here, but there does seem to be an issue with adding the conference alias. Basically there's a lot happening in these Awake and Start methods that could be problematic.