finetune for other languages?

BlinkDL / RWKV-LM

RWKV is an RNN with transformer-level LLM performance. It can be directly trained like a GPT (parallelizable). So it's combining the best of RNN and transformer - great performance, fast inference, saves VRAM, fast training, "infinite" ctx_len, and free sentence embedding.

Apache License 2.0

12.32k stars 838 forks source link

finetune for other languages? #127

Closed batman-do closed 1 year ago

batman-do commented 1 year ago

how I can fine-tune for vietnamese for dialog chatbot ? Thanks guys

XuZhengzhuo commented 1 year ago

Could this possibly work? https://github.com/Blealtan/RWKV-LM-LoRA LoRA fine-tuning

Triang-jyed-driung commented 1 year ago

Use RWKV-4-world

BlinkDL commented 1 year ago

guide: https://mysymphony.jp.net/a/rwkv-character/