search

SeldonIO / seldon-core

An MLOps framework to package, deploy, monitor and manage thousands of production machine learning models
https://www.seldon.io/tech/products/core/
Other
4.37k stars 831 forks source link

Allow configurable upstream timeout for envoy #5098

Open ukclivecox opened 1 year ago

ukclivecox commented 1 year ago

Presently hardwired to 60s. This could initial be changed to 0 to disable timeout. Ideally, a setting per Model/Pipeline.

mafs12 commented 1 year ago

@cliveseldon how can I currently increase that timeout for all requests to all Models? Env var in the envoy pod? And which timeout are we talking about when model takes too long to respond?

mafs12 commented 1 year ago

Found it: https://github.com/SeldonIO/seldon-core/blob/v2.7.0-rc1/scheduler/pkg/envoy/resources/resource.go#L58. Already set to 0.