WisdomShell/kieval: A Knowledge-grounded Interactive Evaluation Framework for Large Language Models
Snippet
A knowledge-grounded interactive evaluation framework for large language models.
README
Description
kieval is a framework for interactively evaluating the capabilities of large language models (LLMs) in a knowledge-grounded setting. It allows users to engage in natural conversations with an LLM, while providing the model with relevant background knowledge to aid in its responses. kieval supports a variety of knowledge sources, including Wikipedia, academic papers (via arXiv), and custom knowledge bases.
Key Features
Interactive Evaluation: Users can engage in free-form conversations with the LLM, exploring its capabilities in a natural setting.
Knowledge Grounding: The LLM has access to relevant background information to help inform its responses, improving its performance.
Flexible Knowledge Sources: Supports integration with various knowledge sources, including Wikipedia, arXiv, and custom knowledge bases.
Configurable Prompts: Users can customize the prompts used to initialize the conversation and provide context to the LLM.
Extensible Architecture: The framework is designed to be easily extensible, allowing for the integration of new LLMs and knowledge sources.
Usage
To use kieval, simply install the package and run the provided script:
pip install kieval
kieval
This will launch the interactive evaluation interface, where you can begin conversing with the LLM and exploring its capabilities.
Contributing
We welcome contributions to the kieval project. If you're interested in helping to develop new features, improve the existing codebase, or address any issues, please check out the contribution guidelines for more information.
ShellLM
commented
3 weeks ago
WisdomShell/kieval: A Knowledge-grounded Interactive Evaluation Framework for Large Language Models
Snippet
A knowledge-grounded interactive evaluation framework for large language models.
README
Description
kievalis a framework for interactively evaluating the capabilities of large language models (LLMs) in a knowledge-grounded setting. It allows users to engage in natural conversations with an LLM, while providing the model with relevant background knowledge to aid in its responses.kievalsupports a variety of knowledge sources, including Wikipedia, academic papers (via arXiv), and custom knowledge bases.Key Features
Usage
To use
kieval, simply install the package and run the provided script:This will launch the interactive evaluation interface, where you can begin conversing with the LLM and exploring its capabilities.
Contributing
We welcome contributions to the
kievalproject. If you're interested in helping to develop new features, improve the existing codebase, or address any issues, please check out the contribution guidelines for more information.Suggested labels
{'label-description': 'Evaluation Framework', 'label-name': 'evaluation-framework', 'gh-repo': 'WisdomShell/kieval', 'confidence': 67.2}