Add Python support for sequence mode

RWKV / rwkv.cpp

INT4/INT5/INT8 and FP16 inference on CPU for RWKV language model

MIT License

1.37k stars 90 forks source link

Closed Mathmagician8191 closed 1 year ago

Mathmagician8191 commented 1 year ago

Added an eval_sequence function to RWKVModel that evaluates a list of tokens at once using sequence mode.

saharNooby commented 1 year ago

@LoganDark plans to rewrite the whole Python API, but I see no problem in having these functions until the rewrite is done.