search

replicate / cog

Containers for machine learning
https://cog.run
Apache License 2.0
8.12k stars 563 forks source link

Use a scope contextvar to to record metrics during a prediction #2047

Closed erbridge closed 2 weeks ago

erbridge commented 2 weeks ago

We use metrics internally to transport metadata about how the model ran out of the model. It's private because it's entirely invisible within our system and we make no claims of stability for how it works.

erbridge commented 2 weeks ago

I like it! What do you make of it?

This works for me, I think.