search

cryscan / web-rwkv

Implementation of the RWKV language model in pure WebGPU/Rust.
Other
210 stars 15 forks source link

1066显卡32G内存报错 #3

Closed kknd222 closed 11 months ago

kknd222 commented 11 months ago

thread 'main' panicked at 'Error in Queue::submit: Validation Error

Caused by: Parent device is lost ', C:\Users\runneradmin.cargo\registry\src\index.crates.io-6f17d22bba15001f\wgpu-0.16.2\src\backend\direct.rs:2252:30 note: run with RUST_BACKTRACE=1 environment variable to display a backtrace CPU 5600X，显卡1060 6G，内存32G D:\dist>chat --model RWKV-4-World-CHNtuned-7B-v1-20230709-ctx4096.st AdapterInfo { name: "NVIDIA GeForce GTX 1060 6GB", vendor: 4318, device: 7171, device_type: DiscreteGpu, driver: "NVIDIA", driver_info: "527.41", backend: Vulkan, } ModelInfo { num_layers: 32, num_emb: 4096, num_vocab: 65536, } 这是爆内存了吗？

cgisky1980 commented 11 months ago

这个配置 3B应该没问题 等一下量化版本吧

cryscan commented 11 months ago

6G独显跑不了7B。

kknd222 commented 11 months ago

这个配置 3B应该没问题 等一下量化版本吧

3B很快，嗖嗖的，就是命令行，输入中文比较麻烦，等API或者GUI

kknd222 commented 11 months ago

6G独显跑不了7B。

好吧，不折腾了。。。

cryscan commented 11 months ago

API可以看 https://github.com/cgisky1980/ai00_rwkv_server