fsprojects / pulsar-client-dotnet

Apache Pulsar native client for .NET (C#/F#/VB)
MIT License
301 stars 47 forks source link

Upgrade AVRO dependencies to 1.11.2 #227

Closed shibd closed 1 year ago

shibd commented 1 year ago

Avro 1.11.2 has been released. It supports logical schemas of C# client.

https://github.com/apache/avro/releases/tag/release-1.11.2

Currently use decimal logic type on schema will throw exception:

Unhandled exception. Avro.AvroException: Unknown schema type: Logical in field DecimalValue
 ---> Avro.AvroException: Unknown schema type: Logical
   at Avro.Reflect.ReflectDefaultWriter.Matches(Schema sc, Object obj)
   at Avro.Specific.SpecificDefaultWriter.WriteUnion(UnionSchema us, Object value, Encoder encoder)
   at Avro.Generic.DefaultWriter.Write(Schema schema, Object value, Encoder encoder)
   at Avro.Reflect.ReflectDefaultWriter.WriteRecord(RecordSchema schema, Object value, Encoder encoder)
   --- End of inner exception stack trace ---
   at Avro.Reflect.ReflectDefaultWriter.WriteRecord(RecordSchema schema, Object value, Encoder encoder)
   at Avro.Generic.DefaultWriter.Write(Schema schema, Object value, Encoder encoder)
   at Avro.Generic.DefaultWriter.Write[T](T value, Encoder encoder)
   at Avro.Reflect.ReflectWriter`1.Write(T value, Encoder encoder)
   at Pulsar.Client.Schema.AvroSchema`1.Encode(T value)

Reproduce code:

    public class DecimalTest
    {
        [AvroSchema("{\n" +
                    "  \"type\": \"bytes\",\n" +
                    "  \"logicalType\": \"decimal\",\n" +
                    "  \"precision\": 28,\n" +
                    "  \"scale\": 8\n" +
                    "}")]
        public AvroDecimal DecimalValue { get; set; }

        public double DoubleValue { get; set; }
        public int IntValue { get; set; }
    }

    internal class Simple
    {
        internal static async Task RunSimple()
        {
            // Arrange
            var testData = new DecimalTest()
            {
                DecimalValue = (decimal)1 / 3,
                DoubleValue = (double)1 / 3,
                IntValue = 1 / 3
            };

            var topicName = $"public/default/test_decimal_precision";
            // Act
            var client = await new PulsarClientBuilder()
                .ServiceUrl("pulsar://127.0.0.1:6650")
                .EnableTls(false)
                .BuildAsync();
            var producer = await client.NewProducer(Schema.AVRO<DecimalTest>())
                .Topic(topicName)
                .ProducerName("test_producer")
                .CreateAsync();
            var consumer = await client.NewConsumer(Schema.AVRO<DecimalTest>())
                .Topic(topicName)
                .SubscriptionName("test_sub")
                .ConsumerName("test_consumer")
                .SubscribeAsync();
            for (int i = 0; i < 10; i++)
            {
              await producer.SendAsync(testData);
            }
            // Act
            var msg = await consumer.ReceiveAsync();
            var receivedTestData = msg.GetValue();

            Console.WriteLine($"After messageId is: '{msg.MessageId}'");
            // Console.WriteLine($"After Decimal data is: '{receivedTestData.DecimalValue}'");

            await client.CloseAsync();
        }
    }
Lanayx commented 1 year ago

I've upgraded dependencies, however this was not really necessary, since newer dependencies work just fine (they can be upgraded without Pulsar.Client upgrade)