[POC] Create a notebook that shows different metrics to evaluate the LLMs

redhat-et / foundation-models-for-documentation

Improve ROSA customer experience (and customer retention) by leveraging foundation models to do “gpt-chat” style search of Red Hat customer documentation assets.

Other

26 stars 12 forks source link

[POC] Create a notebook that shows different metrics to evaluate the LLMs #22

Closed Shreyanand closed 1 year ago

Shreyanand commented 1 year ago

Acceptance

[ ] Notebook that shows how to use evaluation metrics for text generation. Take 2-3 hardcoded examples to show the behavior of each metric. For example, exact match may be very low but BERT score is high because the prediction captures the semantics.

codificat commented 1 year ago

I believe this was completed in #16