search

SillyTavern / SillyTavern-Extras

Extensions API for SillyTavern.
GNU Affero General Public License v3.0
563 stars 133 forks source link

First pass at Apple Silicon MPS back end #66

Closed majick closed 1 year ago

majick commented 1 year ago

This is a quick hack to get the MPS back end for torch up and running, which adds GPU acceleration for Apple Silicon systems. I've been using it for about a week (primarily without SD, but SD works if you have enough memory). A less fussy requirements-macos.txt is included for convenience to get it up and running.

MPS doesn't work well float16, so there's a tweak to make cuda the special case rather than the other way around.

Cohee1207 commented 1 year ago

Tested on Macbook Air M1. Works wonderfully well