RWKV is an RNN with transformer-level LLM performance. It can be directly trained like a GPT (parallelizable). So it's combining the best of RNN and transformer - great performance, fast inference, saves VRAM, fast training, "infinite" ctx_len, and free sentence embedding.
Apache License 2.0
12.32k
stars
838
forks
source link
Trivial Bugfix: not printing deepspeed version if deepspeed has not been imported #132
The check is currently looking for whether the module is installed or not, rather than imported. This makes it impossible to execute non-deepspeed strategies when deepspeed is installed in the environment.
Hello,
Thank you very much for your contributions.
The check is currently looking for whether the module is installed or not, rather than imported. This makes it impossible to execute non-deepspeed strategies when deepspeed is installed in the environment.
~PV