Open Dan-wanna-M opened 1 month ago
Some users might want to mask an array of token ids(e.g. from top_p, top_k) rather than the whole logits. We probably need the caller to provide an output buffer considering how the FFI works.
top_p
top_k
Some users might want to mask an array of token ids(e.g. from
top_p,top_k) rather than the whole logits. We probably need the caller to provide an output buffer considering how the FFI works.