search

iryna-kondr / scikit-llm

Seamlessly integrate LLMs into scikit-learn.
https://beastbyte.ai/
MIT License
2.98k stars 235 forks source link

added crude multi-threading #101

Closed AndreasKarasenko closed 3 weeks ago

AndreasKarasenko commented 3 weeks ago

Adds the option to pass num_workers to predict to speed up inference. Tested for Zero, Few and Dynamic shot classes.

AndreasKarasenko commented 3 weeks ago

I'll add some documentation for this next week.