emit token count metrics and upgrade cog - Githubissues

replicate / cog-triton

A cog implementation of Nvidia's Triton server

Apache License 2.0

11 stars 0 forks source link

emit token count metrics and upgrade cog #41

Closed technillogue closed 3 months ago

technillogue commented 3 months ago

upgrade cog and use new format to specify target concurrency
emit input_token_count and output_token_count metrics