search

HazyResearch / H3

Language Modeling with the H3 State Space Model
Apache License 2.0
515 stars 54 forks source link

CPU Port #7

Open okpatil4u opened 1 year ago

okpatil4u commented 1 year ago

How can I port these models to a CPU only code ? I have 128 GB Mac M1 Ultra Studio system which has 21 Tflops throughput, which render the use of block fftconv moot.

Is it possible to create a CPU only model ? How should one start working on it ?

DanFu09 commented 1 year ago

This is on our roadmap - there's some dependencies that we need to work through for the SSM support (just kernel generation, not even the block FFTConv yet).

Thank you for posting the issue! It's helpful to get a sense of the platforms that people want to run on.

okpatil4u commented 1 year ago

Thanks Dan. After the changes in your roadmap, can we benchmark the huggingface weights on CPU then ?

DanFu09 commented 1 year ago

We’ll be sure that it works!

On Wed, Jan 25, 2023 at 1:34 AM Omkar Patil @.***> wrote:

Thanks Dan. After the changes in your roadmap, can we benchmark the huggingface weights on CPU then ?

— Reply to this email directly, view it on GitHub https://github.com/HazyResearch/H3/issues/7#issuecomment-1403327736, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABDDIIUN6LKV3QS2SQPW5UTWUDXRNANCNFSM6AAAAAAUGBRICE . You are receiving this because you commented.Message ID: @.***>

okpatil4u commented 1 year ago

Hey Dan, do you have any timelines in mind for this ?