search

streamnative / pulsar-client-go

Apache Pulsar Go Client Library
https://pulsar.apache.org/
Apache License 2.0
1 stars 2 forks source link

ISSUE-681: DisableBatching still go thru the batching logic and AUTO_CONSUME can not consume with correct schema type #249

Open sijie opened 2 years ago

sijie commented 2 years ago

Original Issue: apache/pulsar-client-go#681

Actual behavior

When using go client to produce messages with avro schema, and consume using

pulsar-client consume -st auto_consume -s test topic_name -n 0

I'm getting, messages with string schema

----- got message -----                  
key:[Ml8=], properties:[], content:{value=, type=class java.lang.String}
----- got message ----- 
key:[MjNf], properties:[], content:{value=..@, type=class java.lang.String}

This is with or without DisableBatching enabled.

Expected behavior

This is inconsistent with the python client.

When using python, and batching is enabled, I get the above string typed message.

When batching is disabled, I get avro typed message.

I'm not sure how batching has anything to do with AUTO_CONSUME's way of inferring schema type. I expect, whether batching enabled or not, I can always get avro typed messages.

Steps to reproduce

  1. pulsar-client consume -st auto_consume -s test topic_name -n 0
  2. use go client to produce avro messages with DisableBatching=true or DisableBatching=false

It looks like even if DisableBatching is set to true, it still uses a batchBuilder to build a batch, and that batchBuilder respects all the other batching related configs.

https://github.com/apache/pulsar-client-go/blob/efb102522f7c8b25d13c282512fa5a5fe2f2ae44/pulsar/producer_partition.go#L246

Possibly related issue: https://github.com/apache/pulsar/issues/11288

System configuration

Pulsar version: 2.9.0