search

FasterDecoding / REST

REST: Retrieval-Based Speculative Decoding, NAACL 2024
Apache License 2.0
176 stars 11 forks source link

How to support the `repetition_penalty` parameter? #15

Open yangbohust opened 6 months ago

yangbohust commented 6 months ago

I need to connect the REST scheme with the Transformers library. How can I support repetition_penalty parameter through the logits_processor interface?

https://github.com/huggingface/transformers/blob/481a95781404e48b1c80940be17e8279dec82fe8/src/transformers/generation/utils.py#L1735-L1745

By the way, how to make REST support temperature, top_k, top_p parameters through the logits_warper interface?

Thanks~