Serialization of type with multiple dictionaries

[thomas-rabiller]

What is the bug? I am trying to use the serializer for a round trip Serialize/Deserialize of the same C# class type. For most types I've used this has worked fine but with a specific model containing two Dictionary properties, the deserialization fails.

How to reproduce?

This sample code fails using AvroConvert 3.4.6

using SolTechnology.Avro;

public class SampleType
{
    public Dictionary<int, byte[]> ByteArrayData { get; set; } = new();
    public Dictionary<int, float> FloatData { get; set; } = new();
}

var data = new[]
{
    new SampleType {
        FloatData = new Dictionary<int, float>{ [1] = 23.0f } },

    new SampleType {
    ByteArrayData = new Dictionary<int, byte[]>{ [1] = [0,1,2,3,4] } }
};

var file = AvroConvert.Serialize(data);
System.IO.File.WriteAllBytes("test.avro", file);
var read = AvroConvert.Deserialize<SampleType[]>(file);

Exception during Deserialize():

Error: SolTechnology.Avro.Infrastructure.Exceptions.AvroTypeMismatchException: Unable to deserialize [array] of schema [Array] to the target type [Submission#4+SampleType[]]. Inner exception:
---> SolTechnology.Avro.Infrastructure.Exceptions.AvroTypeMismatchException: Unable to deserialize [SampleType] of schema [Record] to the target type [Submission#4+SampleType]. Inner exception:
---> SolTechnology.Avro.Infrastructure.Exceptions.AvroTypeMismatchException: Unable to deserialize [array] of schema [Array] to the target type [System.Collections.Generic.Dictionary`2[System.Int32,System.Single]]. Inner exception:
---> Microsoft.CSharp.RuntimeBinder.RuntimeBinderException: The best overloaded method match for 'System.Collections.Generic.Dictionary<int,float>.Add(int, float)' has some invalid arguments
at CallSite.Target(Closure, CallSite, Object, Object, Object)
at SolTechnology.Avro.AvroObjectServices.Read.Resolver.ResolveDictionary(RecordSchema writerSchema, RecordSchema readerSchema, IReader d, Type type)
at SolTechnology.Avro.AvroObjectServices.Read.Resolver.ResolveArray(TypeSchema writerSchema, TypeSchema readerSchema, IReader reader, Type type, Int64 itemsCount)
at SolTechnology.Avro.AvroObjectServices.Read.Resolver.Resolve(TypeSchema writerSchema, TypeSchema readerSchema, IReader reader, Type type)
--- End of inner exception stack trace ---
at SolTechnology.Avro.AvroObjectServices.Read.Resolver.Resolve(TypeSchema writerSchema, TypeSchema readerSchema, IReader reader, Type type)
at SolTechnology.Avro.AvroObjectServices.Read.Resolver.GetValue(RecordFieldSchema wf, RecordFieldSchema rf, Member memberInfo, IReader dec)
at SolTechnology.Avro.AvroObjectServices.Read.Resolver.ResolveRecord(RecordSchema writerSchema, RecordSchema readerSchema, IReader reader, Type type)
at SolTechnology.Avro.AvroObjectServices.Read.Resolver.Resolve(TypeSchema writerSchema, TypeSchema readerSchema, IReader reader, Type type)
--- End of inner exception stack trace ---
at SolTechnology.Avro.AvroObjectServices.Read.Resolver.Resolve(TypeSchema writerSchema, TypeSchema readerSchema, IReader reader, Type type)
at SolTechnology.Avro.AvroObjectServices.Read.Resolver.ResolveArray(TypeSchema writerSchema, TypeSchema readerSchema, IReader reader, Type type, Int64 itemsCount)
at SolTechnology.Avro.AvroObjectServices.Read.Resolver.Resolve(TypeSchema writerSchema, TypeSchema readerSchema, IReader reader, Type type)
--- End of inner exception stack trace ---
at SolTechnology.Avro.AvroObjectServices.Read.Resolver.Resolve(TypeSchema writerSchema, TypeSchema readerSchema, IReader reader, Type type)
at SolTechnology.Avro.AvroObjectServices.Read.Resolver.Resolve[T](IReader reader, Int64 itemsCount)
at SolTechnology.Avro.Features.Deserialize.Decoder.Read[T](Reader reader, Header header, AbstractCodec codec, Resolver resolver)
at SolTechnology.Avro.Features.Deserialize.Decoder.Decode[T](Stream stream, TypeSchema readSchema)
at SolTechnology.Avro.AvroConvert.Deserialize[T](Byte[] avroBytes)
at Submission#4.<<Initialize>>d__0.MoveNext()
--- End of stack trace from previous location ---
at Microsoft.CodeAnalysis.Scripting.ScriptExecutionState.RunSubmissionsAsync[TResult](ImmutableArray`1 precedingExecutors, Func`2 currentExecutor, StrongBox`1 exceptionHolderOpt, Func`2 catchExceptionOpt, CancellationToken cancellationToken)

Please note that the same code works fine for a class with only one of the two dictionary properties. I suspect there is a mixup between the two during type resolution.

What is the Avro schema?

The schema is auto-generated from this class:

public class SampleType
{
    public Dictionary<int, byte[]> ByteArrayData { get; set; } = new();
    public Dictionary<int, float> FloatData { get; set; } = new();
}

and produces the following json schema (please note the two dictionary types with same name:

{"type":"array","items":{"name":"SampleType","type":"record","fields":[{"name":"ByteArrayData","type":{"type":"array","items":{"name":"KeyValuePair2","namespace":"System.Collections.Generic","type":"record","fields":[{"name":"Key","type":"int"},{"name":"Value","type":"bytes"}]}}},{"name":"FloatData","type":{"type":"array","items":{"name":"KeyValuePair2","namespace":"System.Collections.Generic","type":"record","fields":[{"name":"Key","type":"int"},{"name":"Value","type":"float"}]}}}]}}

What is the Avro data? Fill up the section or provide a sample file

Test avro file produced by the repro code here: test.zip

What is the expected behavior?

Both properties should be deserialized correctly based on the same reader and writer schemas.

[AdrianStrugala]

Hey, Thank you for reporting the bug. I will work on a fix. Best, Adrian

[AdrianStrugala]

Hello @thomas-rabiller

Thank you for your contribution. The fix is available from v.3.4.7 of AvroConvert.

Regards, Adrian

[thomas-rabiller]

Many thanks! I confirm this fixes the problem on my end.

Thanks again for this great library

Kind regards Thomas

[AdrianStrugala]

Hey @thomas-rabiller, I had to change the solution, as the previous introduced breaking changes to other scenarios. Please consider moving to v.3.4.8 Adrian