[Feature Request] Will mojo support M2 ultra to use LLM of 100B paras?

SaraiQX commented 12 months ago

Review Mojo's priorities

[X] I have read the roadmap and priorities and I believe this request falls within the priorities.

What is your request?

As apple-lover without strong CS background, I wish mojo team could work on more work for apple users to use M2 ultra (maybe 192G) in order to tap into the potential of real LLMs (over 100B parameters). I've seen works like vLLM, TGI to speed up the inference of LLM (via cuda).

What is your motivation for this change?

And my imagined scenario include using Apple M2 as the engine and its end devices to build a family-centered LLM-driven app eco. 
Is it possible in near future? Sincere thanks to any masterminds!
Best,
Sarai

Any other details?

No response

lattner commented 12 months ago

Hi @SaraiQX yes it is very possible. We generally don't roadmap out specific narrow features like this, but we know that many folks run on apple silicon (incl myself :-) and that it is an important target to support. Stay tuned for the releases, as we're working hard to get it so you can download mojo. Thanks!

SaraiQX commented 12 months ago

@lattner Hi Mr Lattner, great to have your kind response. I'm definitely interested in any mojo advancement and actually, I've recommended this very young language to coders I know. Wish you all the best! Sarai

modularml / mojo