search

spring-projects / spring-ai

An Application Framework for AI Engineering
https://docs.spring.io/spring-ai/reference/index.html
Apache License 2.0
3.32k stars 850 forks source link

Stability AI does not produce observability metrics #1608

Open habuma opened 3 weeks ago

habuma commented 3 weeks ago

When I generate images with OpenAI, I can see the metrics for those generations in the Actuator's metrics endpoint. But not when generating images with Stability AI. A quick search through the Stability AI code suggests that observability has not been introduced into that model yet.

RodrigoDiasDeOliveira commented 2 weeks ago

Hello whats up?! i think it maybe help, an friend of mine had that same issue, to address this issue, observability needs to be integrated into the Stability AI model to track image generation metrics, similar to how OpenAI's model metrics are visible in the Actuator's metrics endpoint.

Steps to Enable Observability for Stability AI Image Generation Implement Metrics Logging:

Add custom metrics logging for Stability AI image generation requests. This involves defining key performance indicators (KPIs) such as generation time, number of requests, errors, and other relevant metrics to track. Use a monitoring framework like Micrometer or Prometheus, which can be configured to expose metrics through the Actuator’s metrics endpoint. Integrate with Actuator's Metrics Endpoint:

Update the Stability AI service code to push these metrics to the Actuator's metrics endpoint. Ensure that these metrics are tagged appropriately (e.g., stability-ai.image-generation) to distinguish them from other services. Test Observability:

Once observability is implemented, verify that Stability AI image generation metrics appear correctly in the Actuator's endpoint. This will involve generating test images with Stability AI and confirming that metrics data is recorded and visible in your monitoring dashboard. Documentation Update:

Update the project documentation to include the new metrics being tracked and instructions on accessing them. Specify any new configuration steps required for enabling observability in the Stability AI service. By following these steps, you can monitor image generation metrics from Stability AI in a way that aligns with the existing observability setup for OpenAI, providing a comprehensive view of performance across both models.

i hope it can helps