AdrianStrugala / AvroConvert

Rapid Avro serializer for C# .NET
Other
97 stars 27 forks source link

Reduce allocations of Record/Array readers #137

Closed gmanvel closed 7 months ago

gmanvel commented 8 months ago

This PR reduces allocations for Record & Array readers. While profiling memory footprint of deserialization

Fixture fixture = new Fixture();
var data = fixture
    .Build<User>()
    .With(u => u.Offerings, fixture.CreateMany<Offering>(50).ToList)
    .CreateMany(1000)
    .ToArray();

var serialized = AvroConvert.Serialize(data);

Console.WriteLine($"Serialized {serialized.Length}");
Console.ReadLine();

var deserialized = AvroConvert.Deserialize<User[]>(serialized);

Console.WriteLine($"Deserialized {deserialized.Length} users");

noticed few allocations that can be removed

image

In particular closure & ReadOnlyCollection<string> allocations from ResolveRecord

image

Avoid resizing list by setting its initial capacity on creation

image

As a result memory footprint improved

image

and ~ 9% improvement in Deserialize execution

Method Mean Error StdDev Allocated
Deserialize 62.26 ms 1.244 ms 1.574 ms 30.76 MB

VS

Method Mean Error StdDev Allocated
Deserialize 56.59 ms 1.108 ms 1.820 ms 27.34 MB

For reference, benchmark code

[MemoryDiagnoser(displayGenColumns: false)]
[HideColumns("RatioSD")]
public class ReaderBenchmarks
{
    private byte[] _serializedUsers;

    [GlobalSetup]
    public void Setup()
    {
        var fixture = new Fixture();
        var data = fixture
            .Build<User>()
            .With(u => u.Offerings, fixture.CreateMany<Offering>(50).ToList)
            .CreateMany(1000)
            .ToArray();

        _serializedUsers = AvroConvert.Serialize(data);
    }

    [Benchmark]
    public int Deserialize() => AvroConvert.Deserialize<User[]>(_serializedUsers).Length;
}