Add M-series/MPS support

hyperonym / basaran

Basaran is an open-source alternative to the OpenAI text completion API. It provides a compatible streaming API for your Hugging Face Transformers-based text generation models.

MIT License

1.29k stars 81 forks source link

Add M-series/MPS support #172

Closed fakerybakery closed 1 year ago

fakerybakery commented 1 year ago

Thank you for your great repository! This pull request adds support for M1/M2 devices with MPS, if available.

fakerybakery commented 1 year ago

Never mind, this PR seems to raise the following error: RuntimeError: Placeholder storage has not been allocated on MPS device!