Video Chat Recording?

[legistek]

This is fantastic! I can't find any reference to recording though. Is that going to be available? That would be essential if I were to use this in my application.

It would also be convenient - but not critical - if it were possible to record each participant's audiovideo feed separately. At minimum I would like to have each person's audio in a separate channel because we would want to run it through Cognitive Services to get a speech to text transcript with speakers identified.

[jsandoval-msft]

Hi @legistek , the capability described is not currently supported as a feature. However, I've added your input into our feature prioritization process to help us refine our roadmap for new capabilities over the next several months. Thanks!

[anthonysva]

@legistek - can you describe the desired functionality in a bit more detail? For example, bring your own storage, recording retention expectations, configuration and/or policies expected, etc.

[legistek]

Hi @anthonysva happy to and sorry for the lag.

Essentially we need server-side recording of the meeting that's initiated, paused, and completed by API calls. either from our server or a privileged participant.

For our case specifically we really would prefer just getting the raw feeds from each participant as .WEBM streams, just as can be done with WebRTC. That plus a way to know which stream came from which participant, and a means of synchronizing the timecodes, would be all we'd need. (We have a whole code library built atop libav/ffmpeg that can handle converting to MP4, multiplexing ,etc.).

If you wanted to be comprehensive, I imagine most other users would prefer a single recording post-multiplexing, like Teams gives you now, which I guess is where options for resolution might be useful but I can't imagine what beyond that would be needed.

For storage, I figure we could specify an Azure Storage Blob and a connection string and your system could just output everything there, leaving retention, etc. to the user. Assuming data is recorded in batches, I would also love a way to get a callback or to be able to periodically poll to find out when new batches are ready, since like I said I would ultimately like to feed the audio to Cognitive Services to get a realtime transcript with speakers identified.

For policies, I confess I need to study this SDK in more detail so I don't speak without knowing what I'm talking about, but I assume all API calls to this service either come from our server using a privileged key or there is a privileged access tier that we could grant to certain participants that would have the rights to start/pause/terminate the recordings.

On a related note, we're part of MS for Startups and had an app consult with one of your engineers back in July and we talked about this exact scenario and how it fit with our Azure application. We were hoping to essentially put Teams in our web app rather than the other way around but it didn't sound possible at the time (I assume he either wasn't aware or wasn't able to share with me the news about this SDK). Would you guys mind if I PMed you so I could go into more depth about our use case than I'd prefer to on a public forum?

Thanks!

[xtianus79]

@legistek this

[plato001ca]

Hi @anthonysva

As earlier legistek mentioned the recording feature is such an important part of any new applications that we want to build on top of the ACS, the goal in our use case is to process the video stream either in offline or online mode, as part of online mode we are looking to add some extra content on the live stream and also we are looking to detect some specific movement of participants in the video chat. if ACS can provide the live stream through the api or store it somewhere in the azure blob or let us to setup an online storage like like Dropbox, one drive that should be great feature!

[anthonysva]

@legistek - please DM me with more details

[anthonysva]

@plato001ca - you mention "live stream" are you seeking to access and modify a real-time video stream or access the recording and modify the stream or streams? Please elaborate on the live stream capabilities sought or DM me if you prefer. Thank you again for your feedback!

[plato001ca]

Hi Anthony,

Thank you for reaching out to me! regarding the live stream we are thinking about both options 1) access and modify a real-time video stream for either of participants, let's say if I have 3 people/camera in the chat I want to modify ( adding a specific object ( chart, shape, gif) or detect the motions of some objects in the video also detect specific noise ( with proper algorithm/AI). 2) having access to the recorded video with 10/30 second delay, not after the end of call , in our case we are looking to give some advices to the participants based on conversation and also based off the environment activities, imaging if there are couple of people in the background of one of participants we do like to make some action or pushing event.messaging in the smart way. 3) having access on the recorded files ( as whole conversation in one file, each participant separately) , we are thinking to use the cognitive and also GPT3 api for some activities in the background

btw, another important need that we are thinking as part of telephony feature to let to add another participants by phone call ( organizer call someone to join not the other way around), let's say if we have the phone # of participants in our application we want to somehow be able from our office/ enterprise dedicated phone number call the participants mobile/phone.

Please feel free to let me know if you have any questions regarding these topics, I'll be happy to help and contribute more. I hope these services are available asap which we could react to and make a long term plan for the market need.

Best,

On Thu, Oct 15, 2020 at 10:42 PM Anthony Valentino notifications@github.com wrote:

@plato001ca https://github.com/plato001ca - you mention "live stream" are you seeking to access and modify a real-time video stream or access the recording and modify the stream or streams? Please elaborate on the live stream capabilities sought or DM me if you prefer. Thank you again for your feedback!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Azure/Communication/issues/23#issuecomment-709805433, or unsubscribe https://github.com/notifications/unsubscribe-auth/AF3GXF7DANMISXLCAU4HTWDSK7MNNANCNFSM4RXESTGA .

[plato001ca]

@plato001ca - you mention "live stream" are you seeking to access and modify a real-time video stream or access the recording and modify the stream or streams? Please elaborate on the live stream capabilities sought or DM me if you prefer. Thank you again for your feedback!

@anthonysva I did not find any way to directly message/email you ?

[plato001ca]

@anthonysva , is t here any way to email you directly, I want to share all the details with you if it's possible?

[tophpalmer]

Hey @plato001ca , can you please hit up the team over email at acsfeedback@microsoft.com. I'm also going to close this issue but we are tracking the scenario as a feature enhancement.