search

DependencyTrack / hyades

Incubating project for decoupling responsibilities from Dependency-Track's monolithic API server into separate, scalable services.
https://dependencytrack.github.io/hyades/latest
Apache License 2.0
61 stars 18 forks source link

Look into handling unavailable kafka service/ broker #305

Open mehab opened 1 year ago

mehab commented 1 year ago

If for some reason, the topics that the application wants to send data to are temporarily unavailable then the application should be able to hold on to the events for a certain time and retry like a polling mechanism until the topics are available again. This would help in stopping any events being lost due to unavailability of service.

nscuro commented 1 year ago

I think this should be covered by the Kafka clients automatically, see https://www.conduktor.io/kafka/kafka-producer-retries and #70 (section "Producers").

Granted, we'll probably need to tweak and verify the configuration.

nscuro commented 1 year ago

ToDo: Provide some sane defaults (e.g. reducing retry attempts to something less than MAX).