hortonworks / registry

Schema Registry
Apache License 2.0
15 stars 8 forks source link

The registry doesn't create a new schema version if the field's logicalType changed #497

Open rvh opened 6 years ago

rvh commented 6 years ago

Entry point: SchemaRegistryResource.addSchemaVersion

Step 1: create a schema containing the field: { "name": test, "type": "bytes" }

Step 2: create a new version of the schema, updating the field to: { "name": test, "type": "bytes", "logicalType": "decimal", "precision": 4, "scale": 2 }

Expected: Schema Registry creates a new schema version, with the updated field.

Problem: When only the logicalType section of the avro schema changes, the registry does not create a new schema version.

Cause: When submitting a new version, the registry searches the schema version (SchemaVersionLifecycleManager.addSchemaVersion) and create a new version only if the schema version was not found. Because AvroSchemaProvider.build does not include the logicalType attribute of the field in the built string, the fingerprint of the schema does not change when the logicalType changes (same for precision and scale attribute for logicalType decimal). So he finds the previous version of the schema and does not create a new schema version.

ivangreene commented 5 years ago

I’ve noticed this is also an issue when “props” (properties of the schema definition without semantic meaning to Avro but which become available for inspection at runtime) are added/modified/removed from a Schema. I have not yet looked into it much but if this is the only place they are evaluated as being changed we could probably get in a small low impact fix

cgumpert commented 5 years ago

Do you accept PRs on this topic? I would be happy to give it a try to compile a fi but probably need some guidance/review.