star-whale / starwhale

an MLOps/LLMOps platform
https://starwhale.ai
Apache License 2.0
208 stars 35 forks source link

speedup model evaluation in server side #3105

Open tianweidut opened 9 months ago

tianweidut commented 9 months ago

Current Behavior Serialized mode: mode copy -> runtime copy -> runtime restore -> model run

Proposed Behavior Parallized mode: mode copy + runtime restore as the same time.