vespperhq / vespper

Open-source AI copilot that lets you chat with your observability data and code 🧙‍♂️
https://www.vespper.com/?utm_source=github
Apache License 2.0
293 stars 36 forks source link

SigNoz Integration #26

Open david1542 opened 3 months ago

david1542 commented 3 months ago

Add an integration with SigNoz. The integration, in the initial phase, should do the following:

nitin302 commented 3 months ago

Logs API - https://signoz.io/docs/logs-management/logs-api/overview/#api-endpoint Alerts API - maybe instead of Signoz directly, we can integrate with one of the integrations it has via pagerduty/slack - https://signoz.io/docs/setup-alerts-notification/?

nitin302 commented 3 months ago

Also, for Logs API since community edition does not have apikey support, if we can get support via user credentials that would make integration more widely used.

david1542 commented 2 months ago

Thanks for sharing the references to SigNoz docs :) @nitin302 About the alerts API - since we're already integrated with PagerDuty/Opsgenie, it'd indeed be simpler to stick with that. Namely, rely on SigNoz integration with them and integrate with PD/Opsgenie.

About the logs API - can you please elaborate on the API key v.s basic credentials? Are you referring to SigNoz community edition?

BTW - If you want to take a stab at that and implement and initial version of SigNoz integration, that would be amazing :) I can help you on that.

nitin302 commented 2 months ago

Yes, the SigNoz community edition does not have API key support - https://signoz.io/pricing/, but typically user credentials (basic) works.

I might be interested in contributing, if you can share steps... I can create a new PR.

david1542 commented 2 months ago

@nitin302 sure, I can provide some guidance :) Thanks for showing interest.

For the SigNoz integration, I recommend following the structure we used for Coralogix:

Database: Add a new record in the vendors table by updating packages/db/src/models/vendor.ts and including SigNoz in the vendorsData array.

UI Card: Copy the Coralogix UI card in services/dashboard/src/components/Connection/integrations/basic, adjust it for SigNoz, and configure the necessary fields like username, password, and host.

Sub-Agent: Create a sub-agent in Langchain based on the Coralogix structure, modifying it for SigNoz. Currently, the Coralogix agent fetches logs, so we’ll need to decide what SigNoz should do.

(Optional) Processor: If needed, create a processor for Opsgenie incidents to handle SigNoz-related root-cause-analysis flows.

The main work is to create the sub-agent. This is the trickiest part, and requires some prompt engineering. The agent needs to understand the underlying topology, log structures, service names, etc. We did some work on Coralogix and now the agent is in a pretty decent place, but it's still not perfect.

Lemme know what you think about it. I'd be happy to hop on a call with you if you want to discuss further :)

vikash-fourkites commented 2 months ago

@david1542 @nitin302 I am going to attempt Signoz integration. We will see how it turns out. I will update this thread with what I plan and then discuss

david1542 commented 2 months ago

@vikash-fourkites Gotcha, thanks for good spirit of contributing 😊

I know @nitin302 has started working on that. @nitin302 Do you mind sharing your progress maybe?

nitin302 commented 2 months ago

I wasn't able to make much progress but i've provided context to @vikash-fourkites, thanks.

david1542 commented 2 months ago

@nitin302 awesome, thanks a lot. @vikash-fourkites if you need any help, feel free to reach out to me here or on Slack :)

Subash-Mohan commented 1 month ago

Hi @david1542, I noticed there haven't been any updates on the issue since September 18. I'm interested in contributing and would love to take it on if possible. Could you also send me an invitation to join the Slack? Thanks!

david1542 commented 1 month ago

Hi @Subash-Mohan :) Thanks for reaching out! We're currently working on it ourselves. You can join the Slack community using this link. If you want, there are some other issues that you can take.