RWKV is an RNN with transformer-level LLM performance. It can be directly trained like a GPT (parallelizable). So it's combining the best of RNN and transformer - great performance, fast inference, saves VRAM, fast training, "infinite" ctx_len, and free sentence embedding.
What is the RoadMap of RWKV in the near future? Would it remain as a chat language modal and focus on improving its emergent behavior, alignment, and accuracy, or would it evolve into multi modality?
What is the RoadMap of RWKV in the near future? Would it remain as a chat language modal and focus on improving its emergent behavior, alignment, and accuracy, or would it evolve into multi modality?