请问如何搭载state

josStorer / RWKV-Runner

A RWKV management and startup tool, full automation, only 8MB. And provides an interface compatible with the OpenAI API. RWKV is a large language model that is fully open source and available for commercial use.

https://www.rwkv.com

MIT License

5.32k stars 504 forks source link

请问如何搭载state #391

Closed wudisong-hub closed 2 months ago

wudisong-hub commented 2 months ago

我使用的配置是GPU-2G-1B5-World，模型是RWKV-x060-ChnNovel-1B-20240807-ctx4096.st 但是并没有出现state的选择框

yzxh24 commented 2 months ago

不能选 CPU(rwkv.cpp) 那个模式。

josStorer commented 2 months ago

CPU和CUDA这两个模式支持动态state

wudisong-hub commented 2 months ago

好的但是用CPU没有GPU快我尝试下使用CPU