Add `num_proc=4` to Dataset maps in Trainer

tomaarsen / SpanMarkerNER

SpanMarker for Named Entity Recognition

https://tomaarsen.github.io/SpanMarkerNER/

Apache License 2.0

391 stars 27 forks source link

Add `num_proc=4` to Dataset maps in Trainer #50

Open tomaarsen opened 8 months ago

tomaarsen commented 8 months ago

Resolves #47

Hello!

Pull Request overview

Add num_proc=4 to Dataset maps in Trainer

Details

I'm considering allowing users to specify it, but it only makes sense to specify it in TrainingArguments, but I don't have control over that class - it's defined in transformers.

Tom Aarsen