Inference doesn't work on Apple Macbook even when using CPU fp32 as strategy

BlinkDL / ChatRWKV

ChatRWKV is like ChatGPT but powered by RWKV (100% RNN) language model, and open source.

Apache License 2.0

9.43k stars 696 forks source link

Inference doesn't work on Apple Macbook even when using CPU fp32 as strategy #192

Open dodgekaze opened 9 months ago

dodgekaze commented 9 months ago

model = RWKV(model='RWKV-5-World-7B-v2-20240128-ctx4096', strategy='mps fp32')
pipeline = PIPELINE(model, "20B_tokenizer.json") # 20B_tokenizer.json is in https://github.com/BlinkDL/ChatRWKV

ctx = "Assistant: Sure! Here is a very detailed plan to create flying pigs:"
print(ctx, end='')

Outpout:

Assistant: Sure! Here is a very detailed plan to create flying pigs:0029 Comedy[ mah ComedyN Minutes[ mah ComedyN[ mah ComedyNmong[ mah ComedyN ArcherN Dou[ mah ComedyNiduc[ automation agent

BlinkDL commented 9 months ago

use pipeline = PIPELINE(model, "rwkv_vocab_v20230424") for rwkv "world" models