PygmalionAI / data-toolbox

Our data munging code.
GNU Affero General Public License v3.0
34 stars 9 forks source link

LIMARP dataset, first commit #24

Closed TearGosling closed 1 year ago

TearGosling commented 1 year ago

Less is More for Adult Roleplaying dataset. May need work on limiting/dropping entries depending on max tokens, we'll see how much actually compiles. Merging this quick.

TearGosling commented 1 year ago

Still can't test due to my absolutely shattered environment by the way, need to test this on a different computer