continuedev / continue

⏩ Continue is the leading open-source AI code assistant. You can connect any models and any context to build custom autocomplete and chat experiences inside VS Code and JetBrains
https://docs.continue.dev/
Apache License 2.0
17.74k stars 1.38k forks source link

Export metrics to grafana or Prometheus using statsd #1336

Open eladamittai opened 4 months ago

eladamittai commented 4 months ago

Validations

Problem

The telemetry metrics are good, but at least as far as I know, not many people use posthog. I think there could be a more standardized way of exporting metrics in an accessible and flexible way for people to use.

Solution

Use statsd. It's a product that pushes metrics to a time series db. It's right for the plugin usecase because instead of an exporter that needs to know where to pull the metrics from, it pushes them to the db. That way, people can use grafana or other platforms to create their own dashboards based on these metrics. Also, from what I know, statsd is pretty simple to implement, so the overhead of adding it would be rather small.

sestinj commented 4 months ago

Right now we're working on accomplishing this in our commercial product, and plan to at least build a more stable foundation before considering adding this capability to the open-source. Two primary reasons for this are a) open-sourcing is an irreversible process and b) I can foresee a pretty quick proliferation of different integrations that folks might need supported once we allow this.

That said, it'd be quite useful to know more about the kinds of stats/telemetry that you're interested in recording. Are you using PostHog so far via a fork of Continue, or looking for a solution to get started with? If it's within the realm of consideration, or even if you'd just like to hear a bit more, I'd be happy to find time to share what has been built and/or just learn what you need the most