RWKV is an RNN with transformer-level LLM performance. It can be directly trained like a GPT (parallelizable). So it's combining the best of RNN and transformer - great performance, fast inference, saves VRAM, fast training, "infinite" ctx_len, and free sentence embedding.
Apache License 2.0
12.05k
stars
827
forks
source link
python train.py failed in windows11 platform, during ninja build wkv.o #145
1.CUDA\Pytorch\Pytorch-lightning version just match the requirements, but just not working. I'm wondering what kind of GPU do it support.
2.What version should ninja be?
1.CUDA\Pytorch\Pytorch-lightning version just match the requirements, but just not working. I'm wondering what kind of GPU do it support. 2.What version should ninja be?