Support GPU accelerated inference for MacOS on apple silicon chips

FunAudioLLM / CosyVoice

Multi-lingual large voice generation model, providing inference, training and deployment full-stack ability.

https://funaudiollm.github.io/

Apache License 2.0

5.04k stars 512 forks source link

Support GPU accelerated inference for MacOS on apple silicon chips #134

Open limingchina opened 2 months ago

limingchina commented 2 months ago

Is your feature request related to a problem? Please describe. It seems that currently only CPU is used for interference on Mac. I've tried my Mac M3 Pro.

Describe the solution you'd like It would be nice to support GPU accelerated inference for MacOS apple silicon chip.

Additional context Can you list the part where it needs to be modified to support that?

aluminumbox commented 2 months ago

There used to be PR using MPS, but it is slower than cpu on mac, maybe it is not mature yet