aws-samples / aws-virtual-participant-framework-for-rtc

Apache License 2.0
48 stars 11 forks source link

How to run Transcriber app with Kinesis streams #45

Closed tfiechowski closed 10 months ago

tfiechowski commented 10 months ago

Expected behavior and actual behavior

I followed the guide, was able to have the bot join Zoom meeting and run the transcribe_tester_app. I noticed, that no Kinesis video stream is created and going through the codebase, I don't see anything related to Kinesis other than the permissions, eg:

new iam.PolicyStatement({
    actions: [
        "kinesisvideo:CreateStream",
        "kinesisvideo:PutMedia",
        "kinesisvideo:GetDataEndpoint",
        "kinesisvideo:DescribeStream"],
    resources: ['*'],
});

I tried creating Kinesis Stream manually, with the naming convention: zoom-meeting-<meeting-id>-zoom. The convention as mentioned by src/transcribe_tester_app/README.md is:

Note you can look up the Amazon Kinesis Stream *stream-name* above from AWS Console or using the AWS CLI. 
The stream-name prefix is hardcoded as "zoom-meeting-".
This is followed by the 10 digit <zoom-meeting-number> and ending with a suffix that was passed as an environment variable to the Zoom Meeting Windows SDK application `KVS_STREAM_SUFFIX` (see [README](../README.md#set-environment-variables)).

Also, KVS_STREAM_SUFFIX is set to zoom in lib/zoom-meeting-bot-cdk-app-stack.ts:

"KVS_STREAM_SUFFIX": "zoom",

This convention appears to be in line with the Zoom SDK code:

    auto kvs_stream_suffix = std::getenv("KVS_STREAM_SUFFIX");
    std::ostringstream stream_name_ostream;
    stream_name_ostream << "zoom-meeting";

    //If meeting ID is missing passing empty to SNS is better than failing
    ZOOM_SDK_NAMESPACE::IMeetingService* meeting_service = SDKInterfaceWrap::GetInst().GetMeetingService();
    if (meeting_service != NULL && meeting_service->GetMeetingInfo())
    {
        std::ostringstream meeting_number_ostream;
        meeting_number_ostream << meeting_service->GetMeetingInfo()->GetMeetingNumber();
        meeting_number = meeting_number_ostream.str();
        stream_name_ostream << '-' << meeting_number;
    }

    if (kvs_stream_suffix != NULL && strlen(kvs_stream_suffix) > 0)
        stream_name_ostream << '-' << kvs_stream_suffix;

Then, having Fargate task joined the Zoom call successfully, I ran:

node zoom-consumer.js 123123123123 zoom-meeting-123123123123-zoom

The script didn't read anything from stream apparently:

❯ node zoom-consumer.js 123123123123 zoom-meeting-123123123123-zoom
inside zoom-meeting-123123123123-zoom worker
Stream Name: zoom-meeting-123123123123-zoom
inside readKVS worker us-east-1 zoom-meeting-123123123123-zoom
Starting stream zoom-meeting-123123123123-zoom
received endpoint for stream: https://s-xxxxxxx.kinesisvideo.us-east-1.amazonaws.com
Selector:{
  StreamName: 'zoom-meeting-123123123123-zoom',
  StartSelector: { StartSelectorType: 'NOW' }
}
Fragment selector: {"StreamName":"zoom-meeting-123123123123-zoom","StartSelector":{"StartSelectorType":"NOW"}}
done reading
last fragment: zoom-meeting-85172740049-zoom undefined
Last fragment read:  undefined

Which leads me to believe that the task that joined the call doesn't stream the audio/video to the Kinesis. How should I proceed forward? Are there any other repositories missing in the guide? As mentioned before, I don't see any streams created in the CDK code, as well as no commands in the task to create a stream when the task is joining the call. Is this a missing piece in the repo?

Steps to reproduce the problem

Just follow the guide, until the transcribe_tester_app is mentioned to test the transcription.

tfiechowski commented 10 months ago

Okay, it was a silly mistake, I forgot about:

After the virtual participant has joined, allow the virtual participant to locally record the meeting.

When I gave permissions to the recording, everything worked and transcribe_tester_app was showing the live transcription 💪