confluentinc / schema-registry

Confluent Schema Registry for Kafka
https://docs.confluent.io/current/schema-registry/docs/index.html
Other
2.2k stars 1.11k forks source link

Deleted Subject is still detected on registerSchema while absent from getSubjects #2430

Open zaghdous opened 1 year ago

zaghdous commented 1 year ago

Hi, I have a weird behaviour using the schema registry v7.2.0. Some context: We use the abraham-leal/ccloud-schema-exporter to perform backup/restore of schema registry. On restore, the subjects are hard deleted before importing the backup. As result we have a message in the _schema topic with a field deleted=true. After a restart of SR, when we call GET /subjects, the subject is absent which is the expected behaviour.

Bug description:

A client with empty cache tries to register a new schema (for the deleted subject) POST /subjects/mySubject/versions, we expect then that the schema is registered and a new ID is returned. BUT in place, the ID of the deleted schema for mySubject is resolved and we get the error : "i.c.k.s.c.r.e.RestClientException: Overwrite new schema with id 23 is not permitted.; error code: 42205"

The registerSchema operation conflicts between the deleted schema and an existing schema on another subject (same ID 23).

Workaround:

We deleted from the _schemas topic the first schema (deleted=true) by setting the cleanup.policy to "delete" and running kafka-delete-records. After a SR restart, the conflict disapeared and the new schema and we were able to register the new schema with a different ID. => Schema registry still keep a track of deleted schema !

Please let me know if this bug has been resolved in a newer version.

Best regards

OneCricketeer commented 1 year ago

Can you please share your command to reproduce the problem? For example, there's a -syncHardDeletes flag

AishD3 commented 1 year ago

Is this issue still present

shqear93 commented 2 months ago

We had an issue related and maybe same root cause, we had to change the compatibility of the subject to None, then change the mode to READWRITE then create the schema