search

state-spaces / mamba

Mamba SSM architecture
Apache License 2.0
12.97k stars 1.1k forks source link

Question about Apple Silicon and macOS support workaround #130

Open HaskDev0 opened 9 months ago

HaskDev0 commented 9 months ago

Hi all,

As a macOS user running on Apple Silicon I have an interest to use Mamba model on my laptop, but the Linux and NVIDIA requirements do not allow to do this.

Does anyone know how to overcome this problem and if there is a way to train the model and use if for inference of Apple Silicone machines? E.g., rewriting some parts that utilizes Apple GPUs.

I found that there is one attempt to use the model only for inference which uses Rust though:

https://github.com/flawedmatrix/mamba-ssm

Thanks in advance!

albertfgu commented 9 months ago

This is beyond our scope, but hopefully the community will step up here :)

altaic commented 8 months ago

Saw this a few days ago: alxndrTL/mamba.py/mlx

HaskDev0 commented 8 months ago

This is beyond our scope, but hopefully the community will step up here :)

Thank you for commenting back! And still want to say it's a great research and work :)

HaskDev0 commented 8 months ago

Saw this a few days ago: alxndrTL/mamba.py/mlx

Thank you for the resource! I would need to check it on my device too. It's still 2x times slower though... Looks like there is a room for improvement with MLX for Apple then.

altaic commented 8 months ago

Thank you for the resource! I would need to check it on my device too. It's still 2x times slower though... Looks like there is a room for improvement with MLX for Apple then.

Yeah, there are still performance cliffs in mlx, however the author of that mamba repo also notes problems with his implementation in mlx. Unfortunately, there isn't currently a competitive mamba reference implementation using mlx, but I suspect that will change soon 🙂

HaskDev0 commented 8 months ago

Thank you for the resource! I would need to check it on my device too. It's still 2x times slower though... Looks like there is a room for improvement with MLX for Apple then.

Yeah, there are still performance cliffs in mlx, however the author of that mamba repo also notes problems with his implementation in mlx. Unfortunately, there isn't currently a competitive mamba reference implementation using mlx, but I suspect that will change soon 🙂

Yes, would be good to see this :) I might post here a link if I accidentally find something

hg0428 commented 3 weeks ago

Any updates on this? Can we get Apple Silicon support?

fil-does commented 1 day ago

is there a possibility to use mamba with intel (Iris / Xe) ? i can not find any solution for it and i would love to use it in a research project.