search

scaleapi / llm-engine

Scale LLM Engine public repository
https://llm-engine.scale.com
Apache License 2.0
781 stars 55 forks source link

Bump istio proxy memory for gateway #580

Closed yunfeng-scale closed 3 months ago

yunfeng-scale commented 3 months ago

Pull Request Summary

got OOM for large requests on gateway istio containers. 5x memory limit for it

also remove debug logs

Test Plan and Usage Guide

already deployed