Azure / azure-sdk-for-net

This repository is for active development of the Azure SDK for .NET. For consumers of the SDK we recommend visiting our public developer docs at https://learn.microsoft.com/dotnet/azure/ or our versioned developer docs at https://azure.github.io/azure-sdk-for-net.
MIT License
5.17k stars 4.53k forks source link

[BUG] Support complex types in Azure Monitor Query results #37475

Open lmolkova opened 1 year ago

lmolkova commented 1 year ago

Library name and version

Azure.Monitor.Query

Describe the bug

Azure Monitor values can have dynamic type representing arrays or complex objects of arbitrary depth. However when I query with Azure.Monitor.Query, complex types does not seem to be supported.

For example, I can run the following query on our build pipeline data

adx("[https://azsdkengsys.westus2.kusto.windows.net/Pipelines").Build](https://azsdkengsys.westus2.kusto.windows.net/Pipelines%22).Build) 
| where QueueTime >= ago(10d)
| where DefinitionName == "python - confidentialledger - tests"                                                   // just some random thing
| summarize completed = make_list(BuildId) by OrganizationName, RepositoryId, DefinitionName
| take 1

and would get the response like this:

image

Now if I run this query with the library

var res = await client.QueryWorkspaceAsync("c9d2b89f-64d6-4d4d-a9c5-eb42431124ab",
    query,
    new QueryTimeRange(DateTimeOffset.UtcNow.AddDays(-10), DateTimeOffset.UtcNow));

Console.WriteLine(JsonConvert.SerializeObject(res.Value.Table));

I won't get completed array

[["azure-sdk","Azure/azure-sdk-for-python","python - confidentialledger - tests",{}]]

Expected behavior

complex values are deserialized

Actual behavior

complex values are not deserialized

Reproduction Steps

var client = new LogsQueryClient(new DefaultAzureCredential());
string query = @"adx(""https://azsdkengsys.westus2.kusto.windows.net/Pipelines"").Build 
| where QueueTime >= ago(10d)
| where DefinitionName == ""python - confidentialledger - tests""
| summarize completed = make_list(BuildId) by OrganizationName, RepositoryId, DefinitionName
| take 1";

var res = await client.QueryWorkspaceAsync("c9d2b89f-64d6-4d4d-a9c5-eb42431124ab",
    query,
    new QueryTimeRange(DateTimeOffset.UtcNow.AddDays(-10), DateTimeOffset.UtcNow));

Console.WriteLine(JsonConvert.SerializeObject(res.Value.Table.Rows));

I can also define a model and try to deserialize it

var res = await client.QueryWorkspaceAsync<Entry>("c9d2b89f-64d6-4d4d-a9c5-eb42431124ab",
    query,
    new QueryTimeRange(DateTimeOffset.UtcNow.AddDays(-10), DateTimeOffset.UtcNow));

Console.WriteLine(JsonConvert.SerializeObject(res.Value));

class Entry
{
    public long IssueNumber { get; set; }
    public DateTime ClosedAt { get; set; }
    public string OrganizationName { get; set; }
    public string RepositoryId { get; set; }
    public string DefinitionName { get; set; }
    public IEnumerable<int> completed { get; set; }
}

Then I get

Unhandled exception. System.NotSupportedException: The System.Collections.Generic.IEnumerable`1[System.Int32] type is not supported as a deserialization target. Supported types are string, bool, long, decimal, double, object, Guid, DateTimeOffset, TimeRange, BinaryData.
   at Azure.Monitor.Query.RowBinder.TryGet[T](BoundMemberInfo memberInfo, LogsTableRow source, T& value)
   at Azure.Core.TypeBinder`1.BoundMemberInfo`1.Deserialize(TExchange source, Object o, TypeBinder`1 binderImplementation)
   at Azure.Core.TypeBinder`1.BoundTypeInfo.Deserialize[T](TExchange source)
   at Azure.Core.TypeBinder`1.Deserialize[T](TExchange source)
   at Azure.Monitor.Query.RowBinder.BindResults[T](IReadOnlyList`1 tables)
   at Azure.Monitor.Query.LogsQueryClient.QueryWorkspaceAsync[T](String workspaceId, String query, QueryTimeRange timeRange, LogsQueryOptions options, CancellationToken cancellationToken)
   at Program.<Main>$(String[] args) in C:\Users\neska\source\repos\ConsoleApp4\Program.cs:line 16
   at Program.<Main>(String[] args)

Environment

No response

nisha-bhatia commented 11 months ago

This is a known issue that will hopefully be resolved when we complete the public model serialization work item. The goal is to provide a public interface that exposes the currently internal serialization code so customers can access it without needing to use reflection or write their own translation.