Run grid search varying hyperparameters with evaluation

NERC-CEH / llm-eval

Scripts and data for LLM evaluation.

GNU General Public License v3.0

1 stars 2 forks source link

Open matthewcoole opened 1 month ago

matthewcoole commented 1 month ago

To evaluate the most effective model/hyperparam combination we must run a grid search using the evaluation pipeline. This issue depends on #6

matthewcoole commented 1 month ago

Params to vary:

LLMs
- llama3.2/tinyllama (small models)
- llama3.1:8b/70b
- mistral-nemo / mistral-small
- gemma-2:9b/27b
Embeddings model
Chunking
- Chunk length
- Overlap
Prompt