search

redhat-ai-services / ai-accelerator

The AI Accelerator is a template project for setting up Red Hat OpenShift AI using GitOps
28 stars 59 forks source link

Integrate with LLM testing framework such as Phoenix #32

Open diego-torres opened 5 months ago

diego-torres commented 5 months ago

details here: https://docs.arize.com/phoenix RedSAIA project integration: https://gitlab.consulting.redhat.com/redprojectai/infrastructure/appdeploy/-/tree/main/phoenix?ref_type=heads

strangiato commented 2 months ago

So I have been playing around with phoenix and the issue we are going to have with including it in the ai-accelerator is that Phoenix doesn't integrate with the LLM directly, it integrates with tools like Langchain to do telemetry.

We don't have any langchain examples in the ai-accelerator today and I'm not sure that we should extend the scope of it out to that.

I do think that it is an important tool in our toolkit for any LLM related engagement though.