split data set in batches (Run preprocessor in multiple batches after loading the data) - Githubissues

ulf1 / sentence-embedding-evaluation-german

Basically SentEval with German language downstream tasks

Apache License 2.0

0 stars 1 forks source link

split data set in batches (Run preprocessor in multiple batches after loading the data) #51

Open ulf1 opened 1 year ago

ulf1 commented 1 year ago

https://github.com/ulf1/sentence-embedding-evaluation-german/blob/3c7d51f1bd61a4dee5e7fd8401a4e2dce903bca1/sentence_embedding_evaluation_german/data.py#L110