This improves behavior especially if we have lots of HTTP redir checks, none of which can be reached. Currently, we would wait for each of those for 5 minutes (perhaps in parallel) and keep raising workers.
The following configuration demonstrates the issue, assuming packets can be sent to 240.0.0.1 but nothing comes back:
The 10 second timeout is taken from the default in MiniCurl, but note that the meaning is slightly different: that 10-second timeout covers the entire request and response, where this only covers connection. Httplib then defaults to a 5-second timeout per read call; it doesn't seem to have an equivalent "whole-conversation" timeout.
This improves behavior especially if we have lots of HTTP redir checks, none of which can be reached. Currently, we would wait for each of those for 5 minutes (perhaps in parallel) and keep raising workers.
The following configuration demonstrates the issue, assuming packets can be sent to 240.0.0.1 but nothing comes back:
The 10 second timeout is taken from the default in MiniCurl, but note that the meaning is slightly different: that 10-second timeout covers the entire request and response, where this only covers connection. Httplib then defaults to a 5-second timeout per read call; it doesn't seem to have an equivalent "whole-conversation" timeout.