Speech recognition for conjunctions without punctuation to join a long sentence.

[pengcheng933]

    SPXSpeechConfiguration *speechConfig = nil;
    @try {
        NSError *startError2 = nil;
        speechConfig = [[SPXSpeechConfiguration alloc] initWithAuthorizationToken:token region:service error:&startError2];
        if(startError2 != nil){
            NSLog(@"err is %@", startError2);
        }
        [speechConfig setSpeechRecognitionLanguage:lang];
    } @catch (NSException *exception) {
        NSLog(@"Error: %@", exception.reason);
    }

    if (!speechConfig) {
        NSLog(@"Could not load speech config");
        return;
    }
    SPXAudioConfiguration *audioConfig = [[SPXAudioConfiguration alloc] initWithMicrophone:0];
    recognizer = [[SPXSpeechRecognizer alloc] initWithSpeechConfiguration:speechConfig audioConfiguration:audioConfig];

    [recognizer addRecognizedEventHandler:^(SPXSpeechRecognizer *recognizer, SPXSpeechRecognitionEventArgs *eventArgs) {
        NSString *recognizedText = eventArgs.result.text;
        NSLog(@"Final recognized text: %@", recognizedText);
        [speechToTextResult appendFormat:recognizedText];
        self->isRecognize = false;
        [self endIdentification];
    }];
    
    [recognizer addRecognizingEventHandler:^(SPXSpeechRecognizer *recognizer, SPXSpeechRecognitionEventArgs *eventArgs) {
        self->isRecognize = true;
        NSString *intermediateText = eventArgs.result.text;
        NSLog(@"Intermediate recognized text: %@", intermediateText);
    }];

    [recognizer addCanceledEventHandler:^(SPXSpeechRecognizer *recognizer, SPXSpeechRecognitionCanceledEventArgs *eventArgs) {
        NSLog(@"Recognition canceled. Reason: %ld", (long)eventArgs.reason);
        if (eventArgs.errorDetails != nil) {
            NSLog(@"Error details: %@", eventArgs.errorDetails);
            self->isError = true;
            self->isRecognize = false;
        }
    }];
}

Describe the bug

Put this into Deepl and read it out, use the SDK to go to speech to text, the result is missing punctuation, just the hyphenation is not there, I check the API and there is no introduction about this, can the SDK add the hyphenation symbols correctly

To Reproduce

Steps to reproduce the behavior:

1. Put this into Deepl and play it! “Indeed, my MBTI personality is INFJ, so I prefer to stay at home, and I am not very energetic, but I will let myself go out every day, which will make my mood much better.”
2. The result is returned as follows “Indeed, my MBTI personality is INFJ so I prefer to stay at home and I am not very energetic but I will let myself go out every day which will make my mood much better.”, you can see that the punctuation is missing, but not completely.

Version of the Cognitive Services Speech SDK

1.38.0

Platform, Operating System, and Programming Language

• OS: MAC
• Hardware - x64, x86, ARM, ...
• Programming language: Objective-C

[pankopon]

Hi, just to be clear, are you expecting to get a result

• With punctuation exactly like in your Deepl input ("Indeed, my MBTI personality is INFJ, so I prefer to stay at home, and I am not very energetic, but I will let myself go out every day, which will make my mood much better.")? This simply cannot be guaranteed.
• Or with no punctuation at all? This is available as "Lexical" text in detailed JSON results when the recognition output format has been set detailed in speech config (ref. https://learn.microsoft.com/objectivec/cognitive-services/speech/spxoutputformat). The JSON result string can be read from the recognition result object properties with property id SPXSpeechServiceResponseJsonResult (ref. https://learn.microsoft.com/objectivec/cognitive-services/speech/spxpropertyid).

[pengcheng933]

I hope the punctuation can match the Deepl input exactly. If this is too difficult, I would also like the text to be segmented based on speech pauses, as this affects the understanding of the recognized language. Additionally, I have discovered two points:

1.When I switched from Deepl's voice playback to speaking myself, I found that if I paused long enough between phrases, punctuation could be added to segment the text. 2.Does Microsoft's custom speech recognition feature solve my problem? Customized Speech Recognition

[pankopon]

Custom speech may be worth trying in your case, there is information about custom text formatting for the output. Otherwise the only sure way to have punctuation exactly where you want it is to enable dictation and dictate punctuation ("comma", "period" etc.).

[pengcheng933]

Thanks for the reply, will try later!

[github-actions[bot]]

This item has been open without activity for 19 days. Provide a comment on status and remove "update needed" label.

[pankopon]

Closed as answered.