Open Ice3man543 opened 2 years ago
Each template process targets sequentially, so connections to a target remain open in a large number before being reused in another template for the same target. This boils down to a lot of memory usage and so considerable delay between connections to the same target that they become idle/broken. To exploit better connection reuse, we should change the approach and iterate templates over a target rather than iterating targets over a template. In this way, connections would be subsequent and reused without becoming idle/broken.
Having MaxIdleConnsPerHost < 0
already prevents connection pooling
I am not sure if the recent change makes any difference, let me know if I am wrong
connectionConfiguration := &httpclientpool.Configuration{
Connection: &httpclientpool.ConnectionConfiguration{DisableKeepAlive: true},
}
Also having connection pooling enabled is good in terms of performance, maybe clearing the connections in the idle pool
tr := &http.Transport{
IdleConnTimeout: 10 * time.Second, (defaults to 90)
}
or manually clearing the idle connections per template
request.httpClient.HTTPClient.CloseIdleConnections()
would work with a large URL set.
Please describe your feature request:
Recently, we removed default http connection pooling to reduce memory usage. An optional flag should be added to enable that behaviour.
Describe the use case of this feature: