search

iryna-kondr / scikit-llm

Seamlessly integrate LLMs into scikit-learn.
https://beastbyte.ai/
MIT License
3.38k stars 275 forks source link

added crude multi-threading #101

Closed AndreasKarasenko closed 5 months ago

AndreasKarasenko commented 5 months ago

Adds the option to pass num_workers to predict to speed up inference. Tested for Zero, Few and Dynamic shot classes.

AndreasKarasenko commented 5 months ago

I'll add some documentation for this next week.