AdrianStrugala / AvroConvert

Rapid Avro serializer for C# .NET
Other
102 stars 27 forks source link

Unable to Deserialize Avro file to dynamic #147

Closed TurakKatase closed 4 months ago

TurakKatase commented 7 months ago

What is the bug? I have an Avro file generated from an Azure Data Factory Pipeline. When deserializing the file with AvroConvert.Deserialize<List<dynamic>>(avroBytes), I get a list of the correct number of items, but each ExpandoObject is empty.

How to reproduce?

class Program
{
    static void Main(string[] args)
    {
        var avroBytes = File.ReadAllBytes("Sample.avro");

        // Returns a list of the proper length, but all ExpandoObjects in the list are empty
        var dynamicValues = AvroConvert.Deserialize<List<dynamic>>(avroBytes);
    }
}

What is the Avro schema?

{
    "name": "HybridDelivery.ClientLibraryJob",
    "type": "record",
    "fields": [
        {
            "name": "Id",
            "type": [
                "int",
                "null"
            ]
        }
    ]
}

What is the Avro data? Fill up the section or provide a sample file Rename extension to .avro since I couldn't upload the file with an avro extension. Sample.txt

What is the expected behavior? I expected that the result of the deserialization would have an Id property I could access. As I was unable to find any documentation about using the results of the dynamic deserialization, perhaps I'm misunderstanding the use case around it.

AdrianStrugala commented 7 months ago

Hey @TurakKatase, Thank you for reporting the issue. Your understanding is completely right, it's the current limitation with using ExpandoObject in AvroConvert: https://github.com/AdrianStrugala/AvroConvert/issues/107. I will take another look on that, based on the file you've provided, but any idea or PR is more than welcome :) Best, Adrian

TurakKatase commented 7 months ago

It appears my investigation into using the Avro format is at its end, so I probably won't be using the format or AvroConvert regardless of this bug. Turns out Azure Data Factory doesn't support Avro complex types in the schema, removing the usefulness of the format for my use case.

If this is a duplicate of #107, feel free to close this issue.

AdrianStrugala commented 5 months ago

Hey,

Thank you for your contribution. The fix is available from v.3.4.7 of AvroConvert.

Regards, Adrian