search

EmbarkStudios / cervo

Utility wrappers for tract
Apache License 2.0
40 stars 1 forks source link

serve: add loadtest subcommand for use with `cervo serve` or other hosting. #57

Open tgolsson opened 3 weeks ago

tgolsson commented 3 weeks ago

Would be interesting to see how far we can scale cervo serve or other inf servers perf-wise, to avoid needing GPU for distributed serving of small models.