Add and improve scripts for dataset processing

ReaLLMASIC / nanoGPT

The simplest, fastest repository for training/finetuning medium-sized GPTs.

MIT License

23 stars 17 forks source link

Closed klei22 closed 3 months ago

klei22 commented 3 months ago

This adds compatibility layers to:

As well as improvements to mmlu_pro scripts, and jsonl support.

gkielian commented 3 months ago

Lgtm