search

confluentinc / schema-registry

Confluent Schema Registry for Kafka
https://docs.confluent.io/current/schema-registry/docs/index.html
Other
2.23k stars 1.11k forks source link

Schema Registry is continue working even if KafkaStoreReaderThread has been stopped unexpectedly #3192

Open drozd0 opened 3 months ago

drozd0 commented 3 months ago

Hello,

I would raise a question about decision behind keeping Schema Registry healthy when KafkaStoreReaderThread died.

We have a case when Kafka Access Control List(ACLs) was applied and KafkaStoreReaderThread has been stopped due to exception KafkaStoreReader thread has died for an unknown reason

After that Schema Registry may work only in readonly mode with limited state which has been read before the thread crash.

As I understand, only Schema Registry restart may launch the KafkaStoreReaderThread again. If it's true then how to know that the KafkaStoreReaderThread has been stopped to restart the service(k8s pod) again?

We have an health check for /subjects endpoint, I also tried /metadata endpoint and k8s tcp health check on Schema Registry port. But Schema Registry still returns OK response:)

To summarize: Do we really need keep Schema Registry working if KafkaStoreReaderThread died? Or kill the Schema Registry itself to let system know about the failure?

As a suggestion, maybe it's better to kill Schema Registry by throwing FatalExitError here?

What do you think?

Thank you!

drozd0 commented 2 months ago

@rayokota do you have any thoughts? Thank you!