[Question] How does load balancing work for Bot service in Azure?

[neetgupta7]

Hello Since our Bot will cater to needs of thousands of users it will be required for the host environment to support load balancing or be scalable. So I want to understand the options available on Azure. How will multi server configuration affect the session management in case of Bots. What will happen if one message is served to one server and the next to other.

[nwhitmont]

@neetgupta7 For reference, are you using .NET or Node SDK for BotBuilder?

[neetgupta7]

.net

[EricDahlvang]

The BotBuilder is restful and stateless. Statelessness is obtained by a State Service implementation: https://docs.microsoft.com/en-us/bot-framework/dotnet/bot-builder-dotnet-state You should override the default implementation by providing a custom IBotDataStore. There is more information on the State Service here: https://stackoverflow.com/questions/44665826/state-in-botframework/44711219#44711219

From the referenced doc:

By default, the Bot Framework SDK for .NET stores state data using the Bot Framework State service, which is intended for prototyping only and is not designed for use by bots in a production environment. For performance and security reasons in the production environment, you should either use the Bot Builder SDK Azure Extensions to store state data in your own Azure Table storage or Azure DocumentDB store or create a custom implementation of IBotDataStore to store state data in the destination that you specify.

"What will happen if one message is served to one server and the next to other." <-- the bot will respond correctly, since there is no session or state information in the host environment.

---

The Microsoft Bot Framework team prefers that how to questions be submitted on Stack Overflow. The official Bot Framework Github repo  is the preferred platform for submitting bug fixes and feature requests.     I have closed this issue. 

[shahidmh]

I am facing an issue when I manually scale out the app service the conversations are not having a continuity like i designed but when i scale out back to 1 instance it working as expected we are using cosmos dB to store the state what could be the reason for this .how can i do the load balancing here

[SanKumar0632]

.net

Hello, did you find a solution for this??

[SanKumar0632]

The BotBuilder is restful and stateless. Statelessness is obtained by a State Service implementation: https://docs.microsoft.com/en-us/bot-framework/dotnet/bot-builder-dotnet-state You should override the default implementation by providing a custom IBotDataStore. There is more information on the State Service here: https://stackoverflow.com/questions/44665826/state-in-botframework/44711219#44711219

From the referenced doc:

By default, the Bot Framework SDK for .NET stores state data using the Bot Framework State service, which is intended for prototyping only and is not designed for use by bots in a production environment. For performance and security reasons in the production environment, you should either use the Bot Builder SDK Azure Extensions to store state data in your own Azure Table storage or Azure DocumentDB store or create a custom implementation of IBotDataStore to store state data in the destination that you specify.

"What will happen if one message is served to one server and the next to other." <-- the bot will respond correctly, since there is no session or state information in the host environment.

The Microsoft Bot Framework team prefers that how to questions be submitted on Stack Overflow. The official Bot Framework Github repo  is the preferred platform for submitting bug fixes and feature requests.     I have closed this issue.

Hello Eric, When I try to import the Microsoft.Bot.Builder.Dialogs.Internals package to override the IBotDataStore interface, it says that Definition of Internals does not exist in Microsoft.Bot.Builder.Dialogs.

Code:

using Microsoft.Bot.Builder;
using Microsoft.Bot.Builder.Dialogs.Internals;
using MongoDB.Driver;
using System;
using System.Threading.Tasks;

public class MongoDBBotDataStore : IBotDataStore<BotData>
{
    private readonly IMongoCollection<BotData> _collection;

    public MongoDBBotDataStore(IMongoDatabase database)
    {
        _collection = database.GetCollection<BotData>("BotStateCollection");
    }

    public async Task<bool> FlushAsync(IAddress key, CancellationToken cancellationToken)
    {
        // logic to flush bot state data from MongoDB
        throw new NotImplementedException();
    }

    public async Task<BotData> LoadAsync(IAddress key, BotStoreType botStoreType, CancellationToken cancellationToken)
    {
        // logic to load bot state data from MongoDB
        throw new NotImplementedException();
    }

    public async Task SaveAsync(IAddress key, BotStoreType botStoreType, BotData data, CancellationToken cancellationToken)
    {
        // logic to save bot state data to MongoDB
        throw new NotImplementedException();
    }
}