Streamlined Optimization of Deployed Model Configuration via Automated Performance Evaluation / Benchmarking

SeldonIO / seldon-core

An MLOps framework to package, deploy, monitor and manage thousands of production machine learning models

https://www.seldon.io/tech/products/core/

Other

4.37k stars 831 forks source link

Streamlined Optimization of Deployed Model Configuration via Automated Performance Evaluation / Benchmarking #3149

Closed axsaucedo closed 2 years ago

axsaucedo commented 3 years ago

Currently we have examples showing how to leverage vegeta/ghz with argocd to perform grid-search-like performance evaluation https://github.com/SeldonIO/seldon-core/tree/master/examples/batch/benchmarking-argo-workflows, however we would like to have more automated ways that can allow users to perform automated performance evluation to identify the most optimal performance configuration.

ukclivecox commented 2 years ago

Closing for now as probably out of scope for this project