minT(oolkit): Mint awesome, secure and production ready containers just the way you need them! Don't change anything in your container image and minify it by up to 30x (and for compiled languages even more) making it secure too! (free and open source)
Provides an ability to adjust the currently hardcoded http request timeouts. Also need to have corresponding fields in the custom HTTP profile JSON file definitions, request_wait and crawl_request_wait.
The --http-probe-req-wait flag would adjust the timeout value globally for all custom HTTP probe requests including crawler requests.
The --http-probe-crawl-req-wait flag would adjust only the crawler request timeout value (also globally).
Adjust per-request timeouts would require the use of the JSON command file.
Note that per-request timeout adjustments are possible to implement with the --http-probe-cmd flags too, but it would require adjusting the parsing logic for the value, so there's a place for the custom request timeouts. The probe command flag value is already busy as-is though and adding custom request timeouts will make it even messier and harder to use.
kcq
commented
3 months ago
Provides an ability to adjust the currently hardcoded http request timeouts. Also need to have corresponding fields in the custom HTTP profile JSON file definitions,
request_waitandcrawl_request_wait.The
--http-probe-req-waitflag would adjust the timeout value globally for all custom HTTP probe requests including crawler requests.The
--http-probe-crawl-req-waitflag would adjust only the crawler request timeout value (also globally).Adjust per-request timeouts would require the use of the JSON command file.
Note that per-request timeout adjustments are possible to implement with the
--http-probe-cmdflags too, but it would require adjusting the parsing logic for the value, so there's a place for the custom request timeouts. The probe command flag value is already busy as-is though and adding custom request timeouts will make it even messier and harder to use.