microsoft / botbuilder-dotnet

Welcome to the Bot Framework SDK for .NET repository, which is the home for the libraries and packages that enable developers to build sophisticated bot applications using .NET.
https://github.com/Microsoft/botframework
MIT License
872 stars 479 forks source link

Exception - Microsoft.Bot.Builder.AI.Orchestrator.OrchestratorRecognizer: "Failed to find or load Model with path" Exception during initialization: bad allocation #6698

Closed dawwa closed 10 months ago

dawwa commented 12 months ago

After the bot web app running for a few days, started getting this exception. This consistently repros for web app services that have multiple instances, and we have to scale up and down to get rid of the bad nodes. Would you please help to understand what could be the possible reason? and how should we investigate?

Lib ver.: 4.14.1.0

Exception:
Failed to find or load Model with path C:\home\site\wwwroot\generated\orchestratorModels\english

System.InvalidOperationException: 

   at Microsoft.Bot.Builder.AI.Orchestrator.OrchestratorRecognizer.<InitializeModel>b__39_0 (Microsoft.Bot.Builder.AI.Orchestrator, Version=4.14.1.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35) 

   at System.Collections.Concurrent.ConcurrentDictionary`2.GetOrAdd (System.Collections.Concurrent, Version=4.0.15.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a) 

   at Microsoft.Bot.Builder.AI.Orchestrator.OrchestratorRecognizer.InitializeModel (Microsoft.Bot.Builder.AI.Orchestrator, Version=4.14.1.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35) 

   at Microsoft.Bot.Builder.AI.Orchestrator.OrchestratorRecognizer+<RecognizeAsync>d__34.MoveNext (Microsoft.Bot.Builder.AI.Orchestrator, Version=4.14.1.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35) 

   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e) 

   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e) 

   at Microsoft.Bot.Builder.Dialogs.Adaptive.Recognizers.MultiLanguageRecognizer+<RecognizeAsync>d__10.MoveNext (Microsoft.Bot.Builder.Dialogs.Adaptive, Version=4.14.1.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35) 

   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e) 

   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e) 

   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e) 

   at Microsoft.Bot.Builder.Dialogs.Adaptive.Recognizers.CrossTrainedRecognizerSet+<>c__DisplayClass7_0+<<RecognizeAsync>b__0>d.MoveNext (Microsoft.Bot.Builder.Dialogs.Adaptive, Version=4.14.1.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35) 

   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e) 

   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e) 

   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e) 

   at Microsoft.Bot.Builder.Dialogs.Adaptive.Recognizers.CrossTrainedRecognizerSet+<RecognizeAsync>d__7.MoveNext (Microsoft.Bot.Builder.Dialogs.Adaptive, Version=4.14.1.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35) 

   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e) 

   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e) 

   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e) 

   at Microsoft.Bot.Builder.Dialogs.Adaptive.Recognizers.RecognizerSet+<RecognizeAsync>d__6.MoveNext (Microsoft.Bot.Builder.Dialogs.Adaptive, Version=4.14.1.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35) 

   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e) 

   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e) 

   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e) 

   at Microsoft.Bot.Builder.Dialogs.Adaptive.AdaptiveDialog+<OnRecognizeAsync>d__60.MoveNext (Microsoft.Bot.Builder.Dialogs.Adaptive, Version=4.14.1.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35) 

   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e) 

   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e) 

   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e) 

   at Microsoft.Bot.Builder.Dialogs.Adaptive.AdaptiveDialog+<ProcessEventAsync>d__55.MoveNext (Microsoft.Bot.Builder.Dialogs.Adaptive, Version=4.14.1.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35) 

   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e) 

   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e) 

   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e) 

   at Microsoft.Bot.Builder.Dialogs.Adaptive.AdaptiveDialog+<ProcessEventAsync>d__55.MoveNext (Microsoft.Bot.Builder.Dialogs.Adaptive, Version=4.14.1.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35) 

   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e) 

   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e) 

   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e) 

   at Microsoft.Bot.Builder.Dialogs.Adaptive.AdaptiveDialog+<OnPreBubbleEventAsync>d__53.MoveNext (Microsoft.Bot.Builder.Dialogs.Adaptive, Version=4.14.1.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35) 

   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e) 

   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e) 

   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e) 

   at Microsoft.Bot.Builder.Dialogs.Dialog+<OnDialogEventAsync>d__18.MoveNext
dmvtech commented 11 months ago

Hi @dawwa

Do you have simple steps to reproduce? A bot sample as well as configuration settings?

dmvtech commented 10 months ago

Closing due to lack of response.