BlinkDL / RWKV-LM

RWKV is an RNN with transformer-level LLM performance. It can be directly trained like a GPT (parallelizable). So it's combining the best of RNN and transformer - great performance, fast inference, saves VRAM, fast training, "infinite" ctx_len, and free sentence embedding.
Apache License 2.0
12.32k stars 838 forks source link

New Mojo AI Programming Language with RWKV-LM? = Performance Boost? #109

Open Peter2023AI opened 1 year ago

Peter2023AI commented 1 year ago

I was checking into the new Mojo AI programming language, and the possible significant performance boost advantages it might provide to RWKV-LM. Has anyone considered checking into this for RWKV-LM?

Here are the steps according to Mojo’s website to start and finish a project using Mojo:

1) Installation: Download the Mojo distribution from their website and decompress the zip file. Make sure you have installed the required support packages: Tornado, TornadIO2, bcrypt, and wtforms. From the command line in the new folder run python setup.py install. This should install Mojo into your python path.

2) Create a project: To create your first app, you simply need to invoke the mojo_manager application. This will create your project folder with a typical structure that includes a project folder that contains a series of Apps. These apps are independent from one another and offer a way to group different functional areas of your app away into discrete units.

3) Develop your app: Each app has a series of base files that make your app work: models.py for model definition, ui_modules.py for UI Modules for Tornado, urls.py for mapping URLs to relevant Request Handlers, and views.py for request handlers that will manage the various app’s functions.

4) Test and deploy: Once you have developed your app, you can test it locally and then deploy it to a server.

BlinkDL commented 1 year ago

Yeah it looks cool and you are welcome to port RWKV to it

adithyakumars commented 1 year ago

bro the youtube link its show private can u show it bro how to do? pls bro

adithyakumars commented 1 year ago

bro pls bro , bro the youtube link its show private can u show it bro how to do? pls bro @Peter2023AI