Health checks should retry

[jszwedko]

A note for the community

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Problem

Currently, Vector sink health checks only execute once at component start-up and, if they do not succeed, cause Vector to stop if --require-healthy is used. However, there can be transient errors (like https://github.com/vectordotdev/vector/issues/16578) that could be retried by Vector.

We could start with just applying the request options to health checks as well though I could also see users potentially wanting different sets of retry configuration for health checks. I'm also not sure that all sink health checks are request based.

Configuration

No response

Version

vector 0.28.1

Debug Output

No response

Example Data

No response

Additional Context

No response

References

• https://github.com/vectordotdev/vector/issues/16578

[pgassmann]

That's the reason I had to disable the checks for loki sink.

BTW there's a typo in the title: Heath -> Health