Metrics/Logs/Tracers for the User

[jeffchuber]

When the user deploys this service, they will want to:

• capture some level of logging
• understand performance of the service
• capture any bugs that are happening

We should offer some smart defaults around this.

[jeffchuber]

One such example of how to do this in fastapi https://github.com/Blueswen/fastapi-observability

[jeffchuber]

Metrics

• supabse prometheus endpoint and grafana guide - https://supabase.com/docs/guides/platform/metrics
• posthog - https://posthog.com/docs/self-host/configure/monitoring-with-grafana

Logs

• posthog - loki - https://github.com/grafana/loki

I'm asking around in the YC OSS slack community about this

[jeffchuber]

In retrospect, this has more to do with containerization / docker than I realized. I would also really like @levand 's thoughts here!

[jeffchuber]

Logging - skip for now Monitoring - only for the user themselves or, in the case of Chroma, for internal debugging and development. Don't report metrics back to a chroma telemetry service however Traces/Error reporting - we do want to capture anonymized errors.

Tools

• there is a easy and quick sentry & posthog integration I could probably use for errors -- https://posthog.com/tutorials/sentry-plugin-tutorial.... support calling errors back to multiple sentry endpoints so that the user can capture as well. This is probably the easiest lift thing to get off-the-ground. It will play nice with what we are already using (posthog) and avoid a ton of boilerplate or self-hosting snafus.

So I think I should focus this PR on error reporting / traces only for now... Integrate sentry, and make it talk to posthog. Allow the user to add their own sentry api tokens optionally as well.