Usage of speech recognition in a Native AOT library

[WaterBlueNewWorld]

I'm somewhat new in to creating libraries and C#, so i want to know if there is really a possibility for this, but, I want to create a library that can depend on the speech cognitive services in order to use it with FFI through dart code. I've trying by myself using the microsoft docs about the speech SDK and a github repo from a medium article which provides an example on how to make interop with C# and Dart. So far I got this:

using System;
using System.IO;
using System.Threading.Tasks;
using System.Runtime.InteropServices;
using Microsoft.CognitiveServices.Speech;
using Microsoft.CognitiveServices.Speech.Audio;

namespace LibMicCognitive
{
    public class Class1
    {
        static string YourSubscriptionKey = "<MyKey>";
        static string YourServiceRegion = "eastus";

        async static Task<SpeechRecognitionResult> SttTask()
        {
            var speechConfig = SpeechConfig.FromSubscription(YourSubscriptionKey, YourServiceRegion);
            speechConfig.SpeechRecognitionLanguage = "es-MX";

            using var audioConfig = AudioConfig.FromDefaultMicrophoneInput();
            using var speechRecognizer = new SpeechRecognizer(speechConfig, audioConfig);

            Console.WriteLine("Speak into your microphone.");
            var speechRecognitionResult = await speechRecognizer.RecognizeOnceAsync();
            return speechRecognitionResult;
        }

        [UnmanagedCallersOnly(EntryPoint = "voice")]
        public static IntPtr Voice()
        {
            SpeechRecognitionResult res = SttTask().GetAwaiter().GetResult();
            switch (res.Reason)
            {
                case ResultReason.RecognizedSpeech:
                    return Marshal.StringToHGlobalAnsi(res.Text);
                case ResultReason.NoMatch:
                    return Marshal.StringToHGlobalAnsi("Sin Resultados");
                case ResultReason.Canceled:
                    var cancellation = CancellationDetails.FromResult(res);
                    return Marshal.StringToHGlobalAnsi($"Error: {cancellation.Reason}");
                default:
                    return Marshal.StringToHGlobalAnsi($"Error: {res.Reason}");
            }
        }
    }
}

When i compile the project and then try to use the .dll library file on my dart code I get the following error:

Unhandled Exception: System.DllNotFoundException: Unable to load native library 'Microsoft.CognitiveServices.Speech.core.dll' or one of its dependencies.
   at System.Runtime.InteropServices.NativeLibrary.LoadLibErrorTracker.Throw(String) + 0x92
   at Internal.Runtime.CompilerHelpers.InteropHelpers.FixupModuleCell(InteropHelpers.ModuleFixupCell*) + 0x197
   at Internal.Runtime.CompilerHelpers.InteropHelpers.ResolvePInvokeSlow(InteropHelpers.MethodFixupCell*) + 0x44
   at Internal.Runtime.CompilerHelpers.InteropHelpers.ResolvePInvoke(InteropHelpers.MethodFixupCell*) + 0x30
   at Microsoft.CognitiveServices.Speech.Internal.SpeechConfig.speech_config_from_subscription(IntPtr&, String, String) + 0x48
   at Microsoft.CognitiveServices.Speech.SpeechConfig.FromSubscription(String, String) + 0x33
   at LibMicCognitive.Cass1.<SttTask>d__2.MoveNext() + 0x93
--- End of stack trace from previous location ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() + 0x33
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task) + 0xe6
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task) + 0x7b
   at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd(Task) + 0x2c
   at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult() + 0x25
   at LibMicCognitive!<BaseAddress>+0x1ea895

Process finished with exit code -923091058 (0xC8FABF8E)

If there are limitations for this workaround let me know too. Many thanks.

[pankopon]

Hi, I'll need to check if there's someone in our team who knows Dart. Meanwhile,

• What is the environment where your Dart code that calls your C# library runs - is it a standalone console or desktop application on your (Windows) PC, or something else?
• Does this environment have Visual C++ redistributables for VS2019 installed? They are required for Speech SDK dlls.
• Where are the Speech SDK dlls located vs. the application working directory - can they be found according to the standard DLL search order (on Windows)? (Try copying all dlls to the same location where the application is launched from. This would be easier to test and verify with a command-line console application.)
• Can you run some basic Speech SDK C# (.NET Core) quickstart/sample like https://github.com/Azure-Samples/cognitive-services-speech-sdk/tree/master/quickstart/csharp/dotnetcore/from-microphone on the same system successfully?

[WaterBlueNewWorld]

Hi, sorry for the late response, in regards your questions:

• Im using a dart console application to test the sdk.
• I have installed the VS2019 redistributables, in fact the application for what im testing this workaround (a Flutter desktop application) requires the VS2019 C++ redistributable to work.
• Right now, the dlls are located inside a folder called lib, outside that folder the file main.dart is located with the code that calls my dll called LibMicCognitive.dll.
• I already tested the basic tutorial found (here)[https://docs.microsoft.com/en-us/azure/cognitive-services/speech-service/get-started-speech-to-text?tabs=terminal&pivots=programming-language-csharp], and it works flawlessly yet i can try the one you suggest.

Thanks for the response in this matter, I'll update once I test the sample you suggested.

[WaterBlueNewWorld]

Small update on the results from the example you asked me @pankopon: The sample application performed the task without fails.

Also a small side note, the sample app uses .NET 3.1 and mine needs to use .NET 6 in order to compile with AOT. I dont know if this affects on how the sdk works.

[pankopon]

@WaterBlueNewWorld Here's one more thing to try:

• Build a completely stand-alone executable of your Dart application (based on https://dart.dev/tools/dart-compile if valid).
• Place the exe and all dlls (your LibMicCognitive.dll, all Microsoft.CognitiveServices.Speech.* dlls) in the same directory.
• Just in case, also add this directory to Windows PATH.
• Launch a new Command Prompt, cd to the directory in question, and run the executable.

If it still doesn't work then it seems the Speech SDK is not compatible with Dart at the moment. In that case, please consider using one of the supported programming languages for your application instead. Unfortunately we don't have anyone with Dart experience in the Speech SDK team, and there are no plans to develop Dart specific support for now.

[WaterBlueNewWorld]

It worked! Followed your steps and the library worked perfectly Now i only have to handle special characters because some spanish words contain special chararcters but I can handle that on my side, many many thanks @pankopon

[pankopon]

Great to hear you got it working! I'm closing this issue as resolved, please create a new item if further support is needed.