emit token count metrics and upgrade cog - Githubissues

replicate / cog-triton

A cog implementation of Nvidia's Triton server

Apache License 2.0

13 stars 0 forks source link

emit token count metrics and upgrade cog #41

Closed technillogue closed 6 months ago

technillogue commented 6 months ago

upgrade cog and use new format to specify target concurrency
emit input_token_count and output_token_count metrics