patterns-ai-core / langchainrb

Build LLM-powered applications in Ruby
https://rubydoc.info/gems/langchainrb
MIT License
1.45k stars 195 forks source link

Langchain::Evals::CosineSimilarity to calculate simple similarity between actual and expected outputs #855

Open andreibondarev opened 1 month ago

andreibondarev commented 1 month ago

@bborn Okay, I re-packaged it a bit to a small, simple PR. I'm thinking of adding instructions to the README, something like:

# Execute the Assistant.
assistant.add_message_and_run! content: "What is 2+2?"
actual_output = assistant.messages.last.content

# Compare it's output to the expected output
Langchain::Evals::CosineSimilarity.new(llm).score(actual_output: actual_output, expected_output: "4")

( ^^^ Need a better example of course)