feat: Set `workers` through env variable, improved defaults

arcee-ai / fastmlx

FastMLX is a high performance production ready API to host MLX models.

https://arcee-ai.github.io/fastmlx/

Other

227 stars 25 forks source link

feat: Set `workers` through env variable, improved defaults #15

Closed SiddhantSadangi closed 4 months ago

SiddhantSadangi commented 4 months ago

The number of workers can now be specified using the FASTMLX_ENV_VARIABLE. If this is not set, and workers is not provided, it defaults to 2 or the number of CPU cores minus 4, whichever is higher.

⚠️Note: Haven't tested this locally as I don't have a Mac

Blaizzy commented 4 months ago

Hey @SiddhantSadangi

Thanks for the great PR! ❤️

Blaizzy commented 4 months ago

Thanks!

I will merge as soon as the tests clear :)