Closed MohammedAbdulMateen closed 6 years ago
EricDahlvang
commented
6 years ago Yes, you can update userdata externally, and the bot will pick it up when the user communicates with it next. The answer here might help:
The serialization and deserialization of the data field for table storage can be found here: https://github.com/Microsoft/BotBuilder-Azure/blob/0571e46159be56b725e23d87174465fd3a6de632/CSharp/Library/Microsoft.Bot.Builder.Azure/TableBotDataStore.cs
EricDahlvang
commented
6 years ago @MohammedAbdulMateen Do the links provided answer your question?
MohammedAbdulMateen
commented
6 years ago @EricDahlvang thank you, the links were helpful and I was able to pull off what I wanted. I thought of posting the code but didn't get the time
MohammedAbdulMateen
commented
6 years ago I wanted to have my own custom properties in the context.UserData i.e. the Entity (record/row) that gets stored in Azure Table Storage, so that I can use these properties later to perform a particular action in the bot when a user sends a msg.
To do that I followed as shown below,
I implemented a custom class CustomTableBotDataStore for IBotDataStore<BotData> by copying the original TableBotDataStore from source code taken from GitHub and added my custom properties OrgId, AccUserId and Email
Then I created a helper class for Azure Table called AzureTableStorageHelper in the other app where I wanted to set the bot data, I used SetWereSettingsUpdated, DeleteAzureTableRecordsForUser methods to manipulate Azure Table outside bot app.
I set custom properties by calling SetWereSettingsUpdated method and check in bot using CheckWereSettingsUpdated method then call to DeleteAzureTableRecordsForUser
namespace BotApp.App_Start
{
using Domains;
using Microsoft.Bot.Builder.Dialogs;
using Microsoft.Bot.Builder.Dialogs.Internals;
using Microsoft.Bot.Connector;
using Microsoft.WindowsAzure.Storage;
using Microsoft.WindowsAzure.Storage.Table;
using Newtonsoft.Json;
using Newtonsoft.Json.Linq;
using System;
using System.Collections.Generic;
using System.Configuration;
using System.IO;
using System.IO.Compression;
using System.Net;
using System.Threading;
using System.Threading.Tasks;
using System.Web;
using Utilities;
/// <summary>
/// <see cref="IBotDataStore{T}"/> Implementation using Azure Storage Table
/// </summary>
public class CustomTableBotDataStore : IBotDataStore<BotData>
{
private static HashSet<string> checkedTables = new HashSet<string>();
/// <summary>
/// Creates an instance of the <see cref="IBotDataStore{T}"/> that uses the azure table storage.
/// </summary>
/// <param name="connectionString">The storage connection string.</param>
/// <param name="tableName">The name of table.</param>
public CustomTableBotDataStore(string connectionString, string tableName = "botdata")
: this(CloudStorageAccount.Parse(connectionString), tableName)
{
}
/// <summary>
/// Creates an instance of the <see cref="IBotDataStore{T}"/> that uses the azure table storage.
/// </summary>
/// <param name="storageAccount">The storage account.</param>
/// <param name="tableName">The name of table.</param>
public CustomTableBotDataStore(CloudStorageAccount storageAccount, string tableName = "botdata")
{
var tableClient = storageAccount.CreateCloudTableClient();
Table = tableClient.GetTableReference(tableName);
lock (checkedTables)
{
if (!checkedTables.Contains(tableName))
{
Table.CreateIfNotExists();
checkedTables.Add(tableName);
}
}
}
/// <summary>
/// Creates an instance of the <see cref="IBotDataStore{T}"/> that uses the azure table storage.
/// </summary>
/// <param name="table">The cloud table.</param>
public CustomTableBotDataStore(CloudTable table)
{
Table = table;
}
/// <summary>
/// The <see cref="CloudTable"/>.
/// </summary>
public CloudTable Table { get; private set; }
async Task<BotData> IBotDataStore<BotData>.LoadAsync(IAddress key, BotStoreType botStoreType, CancellationToken cancellationToken)
{
var entityKey = BotDataEntity.GetEntityKey(key, botStoreType);
try
{
var result = await Table.ExecuteAsync(TableOperation.Retrieve<BotDataEntity>(entityKey.PartitionKey, entityKey.RowKey));
BotDataEntity entity = (BotDataEntity)result.Result;
if (entity == null)
// empty record ready to be saved
return new BotData(eTag: string.Empty, data: null);
// return botdata
return new BotData(entity.ETag, entity.GetData());
}
catch (StorageException err)
{
throw new HttpException(err.RequestInformation.HttpStatusCode, err.RequestInformation.HttpStatusMessage);
}
}
async Task IBotDataStore<BotData>.SaveAsync(IAddress key, BotStoreType botStoreType, BotData botData, CancellationToken cancellationToken)
{
long orgId;
long userId;
string email;
// set your custom parameter values here
var entityKey = BotDataEntity.GetEntityKey(key, botStoreType);
BotDataEntity entity = new BotDataEntity(key.BotId, key.ChannelId, key.ConversationId, key.UserId, botData.Data, orgId, userId, email)
{
ETag = botData.ETag
};
entity.PartitionKey = entityKey.PartitionKey;
entity.RowKey = entityKey.RowKey;
try
{
if (string.IsNullOrEmpty(entity.ETag))
await Table.ExecuteAsync(TableOperation.Insert(entity));
else if (entity.ETag == "*")
{
if (botData.Data != null)
await Table.ExecuteAsync(TableOperation.InsertOrReplace(entity));
else
await Table.ExecuteAsync(TableOperation.Delete(entity));
}
else
{
if (botData.Data != null)
await Table.ExecuteAsync(TableOperation.Replace(entity));
else
await Table.ExecuteAsync(TableOperation.Delete(entity));
}
}
catch (StorageException err)
{
if ((HttpStatusCode)err.RequestInformation.HttpStatusCode == HttpStatusCode.Conflict)
throw new HttpException((int)HttpStatusCode.PreconditionFailed, err.RequestInformation.HttpStatusMessage);
throw new HttpException(err.RequestInformation.HttpStatusCode, err.RequestInformation.HttpStatusMessage);
}
}
Task<bool> IBotDataStore<BotData>.FlushAsync(IAddress key, CancellationToken cancellationToken)
{
// Everything is saved. Flush is no-op
return Task.FromResult(true);
}
}
internal class EntityKey
{
public EntityKey(string partition, string row)
{
PartitionKey = partition;
RowKey = row;
}
public string PartitionKey { get; private set; }
public string RowKey { get; private set; }
}
internal class BotDataEntity : TableEntity
{
private static readonly JsonSerializerSettings serializationSettings = new JsonSerializerSettings()
{
Formatting = Formatting.None,
NullValueHandling = NullValueHandling.Ignore
};
public BotDataEntity()
{
}
internal BotDataEntity(string botId, string channelId, string conversationId, string userId, object data, long orgId, long accUserId, string email)
{
BotId = botId;
ChannelId = channelId;
ConversationId = conversationId;
UserId = userId;
Data = Serialize(data);
OrgId = orgId;
AccUserId = accUserId;
Email = email;
}
private byte[] Serialize(object data)
{
using (var cmpStream = new MemoryStream())
using (var stream = new GZipStream(cmpStream, CompressionMode.Compress))
using (var streamWriter = new StreamWriter(stream))
{
var serializedJSon = JsonConvert.SerializeObject(data, serializationSettings);
streamWriter.Write(serializedJSon);
streamWriter.Close();
stream.Close();
return cmpStream.ToArray();
}
}
private object Deserialize(byte[] bytes)
{
using (var stream = new MemoryStream(bytes))
using (var gz = new GZipStream(stream, CompressionMode.Decompress))
using (var streamReader = new StreamReader(gz))
{
return JsonConvert.DeserializeObject(streamReader.ReadToEnd());
}
}
internal static EntityKey GetEntityKey(IAddress key, BotStoreType botStoreType)
{
switch (botStoreType)
{
case BotStoreType.BotConversationData:
return new EntityKey($"{key.ChannelId}:conversation", key.ConversationId.SanitizeForAzureKeys());
case BotStoreType.BotUserData:
return new EntityKey($"{key.ChannelId}:user", key.UserId.SanitizeForAzureKeys());
case BotStoreType.BotPrivateConversationData:
return new EntityKey($"{key.ChannelId}:private", $"{key.ConversationId.SanitizeForAzureKeys()}:{key.UserId.SanitizeForAzureKeys()}");
default:
throw new ArgumentException("Unsupported bot store type!");
}
}
internal ObjectT GetData<ObjectT>()
{
return ((JObject)Deserialize(Data)).ToObject<ObjectT>();
}
internal object GetData()
{
return Deserialize(Data);
}
public string BotId { get; set; }
public string ChannelId { get; set; }
public string ConversationId { get; set; }
public string UserId { get; set; }
public byte[] Data { get; set; }
public long OrgId { get; set; }
public long AccUserId { get; set; }
public string Email { get; set; }
}
}namespace AnotherApp.Utilities
{
using Microsoft.Azure;
using Microsoft.Azure.CosmosDB.Table;
using Microsoft.Azure.Storage;
using System;
using System.Collections.Generic;
using System.Linq;
public class AzureTableStorageHelper<T> where T : class, ITableEntity, new()
{
public CloudTable CreateTable(string tableName, string appSettingsKeyName)
{
// Retrieve storage account information from connection string.
CloudStorageAccount storageAccount = CreateStorageAccountFromConnectionString(CloudConfigurationManager.GetSetting(appSettingsKeyName));
// Create a table client for interacting with the table service
CloudTableClient tableClient = storageAccount.CreateCloudTableClient();
// Create a table client for interacting with the table service
CloudTable table = tableClient.GetTableReference(tableName);
try
{
table.CreateIfNotExists();
}
catch (StorageException)
{
throw new StorageException("If you are running with the default configuration please make sure you have started the storage emulator. Press the Windows key and type Azure Storage to select and run it from the list of applications.");
}
return table;
}
public T GetFirstOrDefaultEntity(CloudTable table, string propertyName, string operation, long givenValue)
{
TableQuery<T> query = new TableQuery<T>().Where(TableQuery.GenerateFilterConditionForLong(propertyName, operation, givenValue));
return table.ExecuteQuery(query).FirstOrDefault();
}
public List<T> GetEntities(CloudTable table, string propertyName, string operation, long givenValue)
{
TableQuery<T> query = new TableQuery<T>().Where(TableQuery.GenerateFilterConditionForLong(propertyName, operation, givenValue));
return table.ExecuteQuery(query).ToList();
}
public List<T> GetEntities(CloudTable table, string propertyName, string operation, string givenValue)
{
TableQuery<T> query = new TableQuery<T>().Where(TableQuery.GenerateFilterCondition(propertyName, operation, givenValue));
return table.ExecuteQuery(query).ToList();
}
public T InsertOrReplaceEntity(CloudTable table, T entity)
{
if (entity == null)
{
throw new ArgumentNullException("entity");
}
try
{
// Create the InsertOrReplace table operation
TableOperation insertOrReplaceOperation = TableOperation.InsertOrReplace(entity);
// Execute the operation.
TableResult result = table.Execute(insertOrReplaceOperation);
T insertedEntity = result.Result as T;
return insertedEntity;
}
catch (StorageException)
{
throw;
}
}
public TableResult DeleteEntity(CloudTable table, T deleteEntity)
{
try
{
if (deleteEntity == null)
{
throw new ArgumentNullException("deleteEntity");
}
TableOperation deleteOperation = TableOperation.Delete(deleteEntity);
return table.Execute(deleteOperation);
}
catch (StorageException)
{
throw;
}
}
public void DeleteEntities(CloudTable table, List<T> deleteEntities)
{
try
{
if (deleteEntities == null)
{
throw new ArgumentNullException("deleteEntities");
}
else if (deleteEntities.Count == 0)
{
return;
}
for (int i = 0; i < deleteEntities.Count; i++)
{
TableResult tableResult = DeleteEntity(table, deleteEntities[i]);
}
}
catch (StorageException)
{
throw;
}
}
private CloudStorageAccount CreateStorageAccountFromConnectionString(string storageConnectionString)
{
CloudStorageAccount storageAccount;
try
{
storageAccount = CloudStorageAccount.Parse(storageConnectionString);
}
catch (FormatException)
{
throw new FormatException("Invalid storage account information provided. Please confirm the AccountName and AccountKey are valid in the app.config file.");
}
catch (ArgumentException)
{
throw new ArgumentException("Invalid storage account information provided. Please confirm the AccountName and AccountKey are valid in the app.config file.");
}
return storageAccount;
}
}
} public void SetWereSettingsUpdated(long orgId)
{
var helper = new AzureTableStorageHelper<DynamicTableEntity>();
var table = helper.CreateTable(Constants.App.AzureTableName, Constants.App.AzureStorageConnectionString);
var entities = helper.GetEntities(table, "OrgId", QueryComparisons.Equal, orgId);
for (int i = 0; i < entities.Count; i++)
{
if (entities[i].PartitionKey.Split(':')[1] == "user")
{
dynamic data = GZipSerializationHelper.Deserialize(entities[i].Properties["Data"].BinaryValue);
data.WereSettingsUpdated = true; // WereSettingsUpdated will be the property name to fetch from context.UserData in bot framework application
var binary = GZipSerializationHelper.Serialize(data);
entities[i].Properties["Data"] = EntityProperty.GeneratePropertyForByteArray(binary);
helper.InsertOrReplaceEntity(table, entities[i]);
}
}
} public async Task<bool> CheckWereSettingsUpdated(IDialogContext context, IMessageActivity activity)
{
bool wereSettingsUpdated;
context.UserData.TryGetValue(Constants.BotData.WereSettingsUpdated, out wereSettingsUpdated);
if (wereSettingsUpdated)
{
// call business to delete user data completely using email address
var business = new AnotherAppApi(ConfigurationManager.AppSettings[Constants.Authentication.ApiBaseUrl]);
var email = context.GetEmail(); // context extension
await business.DeleteAzureTableRecordsForUser(email);
// reset conversation stack and bot data
// reset bot data seems to clear not delete the information in "Data" and custom columns only but the records will still be present in Azure Table Storage
// so it is mandatory to call ResetBotData only after calling DeleteAzureTableRecordsForUser
await ResetBotData(activity);
// prompt user to re-login again
await context.PostAsync(RootDialogMsgs.SettingsWereUpdated);
context.Wait(MessageReceived);
return true;
}
return false;
} public void DeleteAzureTableRecordsForUser(string email)
{
var accBusiness = new AccountBusiness();
var userInfo = accBusiness.GetUserId(email);
var azureHelper = new AzureTableStorageHelper<DynamicTableEntity>();
var table = azureHelper.CreateTable(Constants.App.AzureTableName, Constants.App.AzureStorageConnectionString);
var entity = azureHelper.GetFirstOrDefaultEntity(table, "AccUserId", QueryComparisons.Equal, userInfo.AccUserId);
var entities = azureHelper.GetEntities(table, "UserId", QueryComparisons.Equal, entity.Properties["UserId"].StringValue);
azureHelper.DeleteEntities(table, entities);
}
Bot Info
Issue Description
I need to add/update a property for
UserDatain the bot data for a particular user, the bot data is stored in Azure Table Storage, I can retrieve the particular user records using their org id, I've implemented a custom class for Azure Table Storage that derives fromIBotDataStore<BotData>which is storing the user's email, id, org id along with the standard fields provided by the bot framework.For e.g. after retrieving the records I need to add/update a property such as "NotifyToRelogin" set it as "true" or "false" and save it back to Azure Table Storage.
For Azure Table storage I have followed the official documentation to retrieve the records, when trying to convert the binary "Data" property to object I am not able to do, the value looks something like this "H4sIAAAAAAAUBKuuBQBDv6ajAgAAAA=="
I tried the Deserialize method from here but it fails saying "The input stream is not a valid binary format. The starting contents (in bytes) are: ..." may be because of the encoding format used in Azure Table Storage, which I am not aware of.
Code Example
For Azure Table Storage operations I have followed https://github.com/Azure-Samples/storage-table-dotnet-getting-started
Reproduction Steps
1. 2. 3.
Expected Behavior
To add/update properties for a user in bot data in another application so that again it can be accessed in the bot application to perform a specific action or notify the user.
Actual Results
Not able to deserialize the "Data" field to add/update a property and not sure whether this approach is correct i.e. if I successfully update the "Data" as required will I get it back in Bot app using
context.UserData, please suggest a solution for the given requirement.