search

streamnative / pulsar-archived

Apache Pulsar - distributed pub-sub messaging system
https://pulsar.apache.org
Apache License 2.0
73 stars 26 forks source link

ISSUE-13127: sql EOF exception when empty payload message in topic #3360

Open sijie opened 2 years ago

sijie commented 2 years ago

Original Issue: apache/pulsar#13127

Describe the bug Trino EOF exception when I send empty byte array as payload for messages.

To Reproduce Steps to reproduce the behavior:

  1. send some non-empty payload message of avro type
  2. use trino to select * or any select from that topic
  3. trino returns correctly
  4. now, send one empty payload message
  5. use trino to select * or any select from that same topic
  6. trino EOF exception

Expected behavior A clear and concise description of what you expected to happen.

Screenshots error log from trino when using pulsar 2.7.3

org.apache.pulsar.client.api.SchemaSerializationException: java.io.EOFException
        at org.apache.pulsar.client.impl.schema.generic.GenericAvroReader.read(GenericAvroReader.java:106)
        at org.apache.pulsar.client.impl.schema.generic.GenericAvroReader.read(GenericAvroReader.java:42)
        at org.apache.pulsar.client.impl.schema.reader.AbstractMultiVersionReader.read(AbstractMultiVersionReader.java:74)
        at org.apache.pulsar.client.impl.schema.AbstractStructSchema.decode(AbstractStructSchema.java:70)
        at org.apache.pulsar.sql.presto.AvroSchemaHandler.deserialize(AvroSchemaHandler.java:69)
        at org.apache.pulsar.sql.presto.PulsarRecordCursor.advanceNextPosition(PulsarRecordCursor.java:431)
        at io.prestosql.spi.connector.RecordPageSource.getNextPage(RecordPageSource.java:90)
        at io.prestosql.operator.TableScanOperator.getOutput(TableScanOperator.java:302)
        at io.prestosql.operator.Driver.processInternal(Driver.java:379)
        at io.prestosql.operator.Driver.lambda$processFor$8(Driver.java:283)
        at io.prestosql.operator.Driver.tryWithLock(Driver.java:675)
        at io.prestosql.operator.Driver.processFor(Driver.java:276)
        at io.prestosql.execution.SqlTaskExecution$DriverSplitRunner.processFor(SqlTaskExecution.java:1075)
        at io.prestosql.execution.executor.PrioritizedSplitRunner.process(PrioritizedSplitRunner.java:163)
        at io.prestosql.execution.executor.TaskExecutor$TaskRunner.run(TaskExecutor.java:484)
        at io.prestosql.$gen.Presto_332__testversion____20211203_155215_2.run(Unknown Source)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
Caused by: java.io.EOFException
        at org.apache.avro.io.BinaryDecoder.ensureBounds(BinaryDecoder.java:509)
        at org.apache.avro.io.BinaryDecoder.readLong(BinaryDecoder.java:181)
        at org.apache.avro.io.BinaryDecoder.readString(BinaryDecoder.java:279)
        at org.apache.avro.io.ResolvingDecoder.readString(ResolvingDecoder.java:209)
        at org.apache.avro.generic.GenericDatumReader.readString(GenericDatumReader.java:467)
        at org.apache.avro.generic.GenericDatumReader.readString(GenericDatumReader.java:458)
        at org.apache.avro.generic.GenericDatumReader.readWithoutConversion(GenericDatumReader.java:182)
        at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:151)
        at org.apache.avro.generic.GenericDatumReader.readField(GenericDatumReader.java:248)
        at org.apache.avro.generic.GenericDatumReader.readRecord(GenericDatumReader.java:237)
        at org.apache.avro.generic.GenericDatumReader.readWithoutConversion(GenericDatumReader.java:170)
        at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:151)
        at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:144)
        at org.apache.pulsar.client.impl.schema.generic.GenericAvroReader.read(GenericAvroReader.java:101)
        ... 18 more

when using pulsar 2.8.0, it's

java.lang.NullPointerException
        at org.apache.pulsar.sql.presto.PulsarRecordCursor.advanceNextPosition(PulsarRecordCursor.java:493)
        at io.prestosql.spi.connector.RecordPageSource.getNextPage(RecordPageSource.java:90)
        at io.prestosql.operator.TableScanOperator.getOutput(TableScanOperator.java:302)
        at io.prestosql.operator.Driver.processInternal(Driver.java:379)
        at io.prestosql.operator.Driver.lambda$processFor$8(Driver.java:283)
        at io.prestosql.operator.Driver.tryWithLock(Driver.java:675)
        at io.prestosql.operator.Driver.processFor(Driver.java:276)
        at io.prestosql.execution.SqlTaskExecution$DriverSplitRunner.processFor(SqlTaskExecution.java:1075)
        at io.prestosql.execution.executor.PrioritizedSplitRunner.process(PrioritizedSplitRunner.java:163)
        at io.prestosql.execution.executor.TaskExecutor$TaskRunner.run(TaskExecutor.java:484)
        at io.prestosql.$gen.Presto_332__testversion____20211203_181533_2.run(Unknown Source)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

Desktop (please complete the following information):

Additional context Add any other context about the problem here.

github-actions[bot] commented 2 years ago

The issue had no activity for 30 days, mark with Stale label.