Open dsanmart opened 6 months ago
Took a look, a few things.
Linux / mps support looks to be in progress still https://github.com/pytorch/pytorch/issues/81224 so running in a container isn't ready yet.
MPS has some limitations around complex tensors atm. Since gemma uses RoPE, it uses complex tensors and errors out if you run it locally.
Running locally with python scripts/run.py --ckpt gemma-2b-it.ckpt --variant 2b --device mps
pytorch 2.3 has bf16 and complex tensor support, and dockerised containers now work @lamroger
Will there be MPS support for the Gemma models? It would enable access to a larger community.