search

Hydrospheredata / hydro-serving

MLOps Platform
http://docs.hydrosphere.io
Apache License 2.0
271 stars 41 forks source link

Envoy Proxy #248

Closed albertnanda closed 5 years ago

albertnanda commented 5 years ago

Hi,

How can I configure envoy proxy with hydrosphere. Where are the config files.

Thanks, AN

KineticCookie commented 5 years ago

Hi. I'm not sure what kind of configuration do you want. We use our sidecar image with some essential setup here: https://github.com/Hydrospheredata/hydro-serving-sidecar

albertnanda commented 5 years ago

@KineticCookie : I was testing my service with a lot of requests. I want to understand if envoy will automatically spin new containers based on load or should i create duplicate applications and configure it.

KineticCookie commented 5 years ago

Understood. In that case, I need to give a small tour into the specifics of serving. So, manager creates unique container for every model and runtime combination and creates dynamic routing table for every application. The route table is the broadcasted to gateway and envoy services. The actual grpc call to predict the model goes through gateway, which applies additional info and logic to the request and sends it to the further destination described in the tables.

Thus you need to scale gateway and model containers. Envoy is a simple service mesh and doesn't manage containers.