Closed Arnovsky closed 1 month ago
There are other issues that are related to the pushgateway implementation, no keep-alive on the connection, no connection pooling, no batching, we can fix these as well, but not in the context of this PR.
@LucaCanali Is it possible to get a review? Thanks!
Thank you @Arnovsky for your contribution.
Hey, this PR adds the capability to configure the timeout for the PushGateway http connection, we have observed cases where large spark jobs (hundreds - thousands) of stages are lagging due to the generous 5000ms default timeout, our aim is to allow this behaviour to be configurable.
I also fixed some inconsistencies with the formatting, please let me know if this is an issue and you'd like me to revert.