microsoft / BotBuilder-RealTimeMediaCalling

BotBuilder-RealTimeMediaCalling extends the BotBuilder to enable bots to engage in Skype audio-video calling. It provides real-time, programmable access to the voice, video, and screen sharing streams of a Skype call. The bot is a direct participant in a Skype 1:1 call.
MIT License
76 stars 36 forks source link

Screen sharing #37

Closed totalasma closed 6 years ago

totalasma commented 6 years ago

Hi, are there any examples/instructions about how to enable screen-sharing?

ssulzer commented 6 years ago

Hi @totalasma,

You can enable a bot to receive screen sharing video from the caller. Having the bot be the screen sharer is not yet supported.

To add a screen sharing capability, create a receive-only VideoSocket and give it to MediaPlatform.CreateMediaConfiguration( ) as a vbssSocket; e.g.:

            // video socket for "vbss": video-based screen sharing
            _vbssSocket = new VideoSocket(new VideoSocketSettings
            {
                StreamDirections = StreamDirection.Recvonly,  // must be Recvonly
                ReceiveColorFormat = VideoColorFormat.NV12,
                CallId = correlationId
            });

            _vbssSocket.VideoMediaReceived += OnScreenSharingVideoMediaReceived;

            MediaConfiguration = MediaPlatform.CreateMediaConfiguration(
                            _audioSocket,
                            videoSockets: null,  // provide  a VideoSocket for "main" video if you want
                            vbssSocket: _vbssSocket);

Once the caller starts a screen share session, the vbssSocket should start delivering VideoBuffers to the VideoMediaReceived event handler.

totalasma commented 6 years ago

Thanks @ssulzer for advices, got screen share working with "null" videosocket parameter!

Although I did't get screensharing working with videosocket feed at the same time. Videosocket with screen sharing is using a ordered list type of object as parameter. I tried that with huebot and it leads to "you can't talk to this bot just yet, but we're working on it" message. Is videosocket event fired correctly on list (multiple objects subscribing to same event) or could you have example about this aswell? Thanks in advance.

SOLVED this by defining list to fixed array of videosockets.

VideoSocket[] videoSockets = new VideoSocket[1]; videoSockets[0] = _videoSocket;

totalasma commented 6 years ago

Is it possible to save videoSocket and screen sharing socket at the same time?

ssulzer commented 6 years ago

Yes, you can create a VideoSocket for "primary" video and one for screen sharing:

        // video socket for primary video
        _videoSocket = new VideoSocket(new VideoSocketSettings
        {
            StreamDirections = StreamDirection.Sendrecv,
            ReceiveColorFormat = VideoColorFormat.NV12,
            SupportedSendVideoFormats = // list of video send formats 
            CallId = correlationId
        });

        // video socket for "vbss": video-based screen sharing
        _vbssSocket = new VideoSocket(new VideoSocketSettings
        {
            StreamDirections = StreamDirection.Recvonly,  // must be Recvonly
            ReceiveColorFormat = VideoColorFormat.NV12,
            CallId = correlationId
        });

        _videoSocket.VideoMediaReceived += OnPrimaryVideoMediaReceived;
        _vbssSocket.VideoMediaReceived += OnScreenSharingVideoMediaReceived;

        MediaConfiguration = MediaPlatform.CreateMediaConfiguration(
                        _audioSocket,
                        videoSockets: new List<VideoSocket> { _videoSocket },
                        vbssSocket: _vbssSocket);
totalasma commented 6 years ago

@ssulzer videoSockets parameter for MediaPlatform.CreateMediaConfiguration is not working. It suggests that it can not convert System.Collection.List to System.Collection.IList<Microsoft.Skype.Bots.Media.IVideoSocket. I have installed Microsoft.Skype.Bots.Media v 1.8.0.261-alpha.

frleger commented 6 years ago

@totalasma Try creating the video sockets list as a List< IVideoSocket>

ie. new List< IVideoSocket> { _videoSocket }, instead of new List< VideoSocket> { _videoSocket },

totalasma commented 6 years ago

Thanks @frleger that worked. It now opens not video socket and screensharing socket streams.