amazon-connect / ai-powered-speech-analytics-for-amazon-connect

The AI Powered Speech Analytics for Amazon Connect solution provides the combination of speech to text transcription, translation into preferred languages, and insights for agents and supervisors all in real-time
https://aws.amazon.com/solutions/implementations/ai-powered-speech-analytics-for-amazon-connect/
Other
56 stars 54 forks source link

In transcribeStreaming, some languages is not working. #21

Closed nuatmochoi closed 3 years ago

nuatmochoi commented 3 years ago

After communication with AWS Support Center, I made this issue. In transcribeStreaming, languageCode korean(ko-KR) is not working.

When I tested KVSTranscriber in English(en-US) without any settings, There's no problem in process of speech-to-text. In reference of other language (ex> en-GB, es-US), I set the languageCode to ko-KR, many errors occurred like below:

kvsTranscriber Logs (It's too long, so I cut it. If you wanna whole logs, I can reply it)
``` 2021-03-12 04:32:18 INFO KVSUtils:210 - StartSelector set to NOW 2021-03-12 04:32:18 INFO KVSUtils:218 - GetMedia called on stream kinesis-stream-test-connect-gsss-contact-d5ddaae5-9aa5-4288-acc4-2d1599582727 response 200 requestId e2422832-3ede-be7d-ba86-d920ae539d76 2021-03-12 04:32:18 INFO KVSUtils:210 - StartSelector set to NOW 2021-03-12 04:32:19 INFO KVSUtils:218 - GetMedia called on stream kinesis-stream-test-connect-gsss-contact-d5ddaae5-9aa5-4288-acc4-2d1599582727 response 200 requestId d4803654-9f4b-4559-8c44-c7470fca807f 2021-03-12 04:32:21 INFO KVSTranscribeStreamingLambda:159 - Calling Transcribe service.. 2021-03-12 04:32:24 INFO StreamTranscriptionBehaviorImpl:53 - 1615523544849 Received Initial response from Transcribe. Request Id: 52db6586-dd55-4931-9039-ae0ffd3c5c90 2021-03-12 04:32:24 INFO StreamTranscriptionBehaviorImpl:53 - 1615523544869 Received Initial response from Transcribe. Request Id: 8d05bbc8-31bd-4eba-8636-b9fc908b0215 2021-03-12 04:32:40 INFO TranscribeStreamingRetryClient:214 - Reached on error but doing nothingsoftware.amazon.awssdk.services.transcribestreaming.model.BadRequestException: Your request timed out because no new audio was received for 15 seconds. (Service: transcribe, Status Code: 400, Request ID: 8d05bbc8-31bd-4eba-8636-b9fc908b0215, Extended Request ID: null) 2021-03-12 04:32:40 INFO TranscribeStreamingRetryClient:214 - Reached on error but doing nothingjava.util.concurrent.CompletionException: software.amazon.awssdk.services.transcribestreaming.model.BadRequestException: Your request timed out because no new audio was received for 15 seconds. (Service: transcribe, Status Code: 400, Request ID: 8d05bbc8-31bd-4eba-8636-b9fc908b0215, Extended Request ID: null) 2021-03-12 04:32:40 DEBUG TranscribeStreamingRetryClient:164 - Error occurred on channel AUDIO_FROM_CUSTOMER : software.amazon.awssdk.services.transcribestreaming.model.BadRequestException: Your request timed out because no new audio was received for 15 seconds. (Service: transcribe, Status Code: 400, Request ID: 8d05bbc8-31bd-4eba-8636-b9fc908b0215, Extended Request ID: null) java.util.concurrent.CompletionException: software.amazon.awssdk.services.transcribestreaming.model.BadRequestException: Your request timed out because no new audio was received for 15 seconds. (Service: transcribe, Status Code: 400, Request ID: 8d05bbc8-31bd-4eba-8636-b9fc908b0215, Extended Request ID: null) at software.amazon.awssdk.utils.CompletableFutureUtils.errorAsCompletionException(CompletableFutureUtils.java:60) at software.amazon.awssdk.core.internal.http.pipeline.stages.AsyncExecutionFailureExceptionReportingStage.lambda$execute$0(AsyncExecutionFailureExceptionReportingStage.java:51) ```

I thought main problem is: Your request timed out because no new audio was received for 15 seconds. But I certainly speak on the phone. The call test environment for korean was same with english.

I already know some preconditions are necessary for using added supported languages such as korean, japanese. I took those actions.

  1. Set languageCode to korean(ko-KR) in Contact Flows
  2. Set 'Start Media Streaming' block by 'from the customer' (can select both of options - from & to)
  3. Change kvs_Trigger Function for LanguageCode Attributes
  4. Update the newest version of transcribeStreaming sdk in build.gradle

AWS Support Center said "Some changes to the Transcribe SDK in the newer version that supports Korean is likely what is causing the issue". And I agree it. So I want to track the change against this issue. And I want to know what is the origin problem that causes this issue?

Thank you. I'll be waiting the response.

efh365 commented 3 years ago

@nuatmochoi Can you try replacing the code for the KVSTranscriber with the code built in this PR: https://github.com/amazon-connect/amazon-connect-realtime-transcription/pull/57 ? That has updated dependencies and should support Korean. I tested with Australian English, which is a newer supported language, and it worked for me.

nuatmochoi commented 3 years ago

Sorry for late confirm. I tested with your latest PR, using Korean(ko-KR), and it works well. Thank you for your timely and excellent solution to the problem. This issue was solved, and I close this issue.