Add three data loader - Githubissues

WukLab / preble

Stateful LLM Serving

Apache License 2.0

38 stars 6 forks source link

Add three data loader #53

Closed dongmingli-Ben closed 7 months ago

dongmingli-Ben commented 7 months ago

This PR adds three dataloader, including

Agent: Chameleon with TabMWP dataset, 9 system prompts
Agent: CREATOR with MATH dataset, 3 system prompts
Tool: ToolQA, 2 system prompts

The data path for each of these datasets should be:

Chameleon with TabMWP: results/tabmwp/chameleon_gpt4_test_cache.jsonl
CREATOR with MATH: MATH
ToolQA: data/questions