apache / pulsar

Apache Pulsar - distributed pub-sub messaging system
https://pulsar.apache.org/
Apache License 2.0
14.28k stars 3.59k forks source link

[Bug] When the Schema structure is incorrect, SQL queries return Not a record: "string" #20945

Open crossoverJie opened 1 year ago

crossoverJie commented 1 year ago

Search before asking

Version

master

Minimal reproduce step

Create producer

Producer p = pulsarClient.newProducer(Schema.JSON(String.class)).create()

It would generate the following schema:

{
  "version": 0,
  "schemaInfo": {
    "name": "schema-test-2",
    "schema": "\"string\"",
    "type": "JSON",
    "timestamp": 1691379461556,
    "properties": {
      "__alwaysAllowNull": "true",
      "__jsr310ConversionEnabled": "false"
    }
  }
}

And query from Trino:

select * from pulsar."public/default"."schema-test-2";

image

Trino service did not report any exception. image

What did you expect to see?

Query success.

What did you see instead?

Query 20230807_075157_00000_b7hfa failed: Not a record: "string"

Anything else?

This exception was thrown from there:

https://github.com/apache/pulsar/blob/2ab184e49a036a1dd10dc537bef4ab034a5ad5e0/pulsar-client/src/main/java/org/apache/pulsar/client/impl/schema/generic/GenericSchemaImpl.java#L43-L46

Because there are a large amount of these types of codes in our production environment, we hope that they can be compatible in this situation in the pulsar-sql module.

When parsing the Schema encounters exceptions, use defaultSchema to regenerate, like this:

image

Are you willing to submit a PR?

github-actions[bot] commented 1 year ago

The issue had no activity for 30 days, mark with Stale label.