search

abacaj / mpt-30B-inference

Run inference on MPT-30B using CPU
MIT License
572 stars 94 forks source link

Can we use GPU inference? #9

Closed wajihullahbaig closed 1 year ago

wajihullahbaig commented 1 year ago

Thank you for such a great repo. I was wondering if we can use GPU inference for text generation?

abacaj commented 1 year ago

Not at the moment, issue tracking it here https://github.com/ggerganov/ggml/issues/230