search

Aiven-Open / karapace

Karapace - Your Apache Kafka® essentials in one tool
https://karapace.io
Apache License 2.0
470 stars 71 forks source link

chore, schema-coordinator: handle `aiokafka` retriable errors #934

Closed nosahama closed 2 months ago

nosahama commented 2 months ago

About this change - What it does

Handle aiokafka retriable errors, we see from here the coordinator.check_errors() is used in the consumer when getting the position, so this will always raise the errors that are registered with coordinator._push_error_to_user(), that way we do not flood application monitoring systems and instead log a warning.

Service logs

Showing the now warning logs for the GroupCoordinatorNotAvailableError, we can also add a catch for UnknownTopicOrPartitionError which gets raised here.

Screenshot 2024-08-20 at 15 23 05

We also reduce the log level for the master_coordinator bootstrap errors as they [will be retried] (https://github.com/Aiven-Open/karapace/blob/main/karapace/coordinator/master_coordinator.py#L54)

Screenshot 2024-08-20 at 15 44 23
github-actions[bot] commented 2 months ago

Coverage report

Click to see where and how coverage changed

FileStatementsMissingCoverageCoverage
(new stmts)		Lines missing
  karapace/coordinator
  master_coordinator.py 53
  schema_coordinator.py
Project Total  

This report was generated by python-coverage-comment-action

nosahama commented 2 months ago

Closing this PR, the work will be consolidated into #936 and also we only use the AdminClient from aiokafka, so ideally the check_errors() function is not called, so this is not the right place to intercept the error.