Giskard-AI / giskard

🐢 Open-Source Evaluation & Testing for ML models & LLMs
https://docs.giskard.ai
Apache License 2.0
3.92k stars 250 forks source link

Fixed issue of RAG results plots rendering #2036

Open kevinmessiaen opened 1 day ago

kevinmessiaen commented 1 day ago

Description

Bokeh >= 3.6 cause issue with rendering plots when saving to html file

How to reproduce

Run this code

import os

import giskard
from giskard.llm.client.openai import OpenAIClient

os.environ["OPENAI_API_KEY"] = "sk-..."

giskard.llm.set_llm_api("openai")
oc = OpenAIClient(model="gpt-4-turbo-preview")
giskard.llm.set_default_client(oc)

import pandas as pd
from giskard.rag import KnowledgeBase, generate_testset

# Load your data and initialize the KnowledgeBase
df = pd.read_csv("knowledge_base.csv")

knowledge_base = KnowledgeBase.from_pandas(df, columns=["q", "a"])

# Generate a testset with 10 questions & answers for each question types (this will take a while)
testset = generate_testset(
    knowledge_base,
    num_questions=10,
    language='en',  # optional, we'll auto detect if not provided
    agent_description="A customer support chatbot for company X", # helps generating better questions
)

# Save the generated testset
testset.save("my_testset.jsonl")

# You can easily load it back
from giskard.rag import QATestset

loaded_testset = QATestset.load("my_testset.jsonl")

# Convert it to a pandas dataframe
df = loaded_testset.to_pandas()

#%%
from giskard.rag import evaluate

# Wrap your RAG model
def get_answer_fn(question: str, history=None) -> str:
    """A function representing your RAG agent."""
    # Format appropriately the history for your RAG agent
    messages = history if history else []
    messages.append({"role": "user", "content": question})

    # Get the answer
    answer = "I don't know"  # could be langchain, llama_index, etc.

    return answer

# Run the evaluation and get a report
report = evaluate(get_answer_fn, testset=testset, knowledge_base=knowledge_base)

report.to_html('test.html')

Open the test.html generated file to see that the plot is not rendering when bokeh>3.6

Type of Change

sonarcloud[bot] commented 1 day ago

Quality Gate Passed Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarCloud