search

FasterDecoding / REST

REST: Retrieval-Based Speculative Decoding, NAACL 2024
Apache License 2.0
176 stars 11 forks source link

Could REST support beam search sampling policy? #2

Closed leiwen83 closed 7 months ago

leiwen83 commented 12 months ago

Hi,

I wonder to know whether REST could support beam search.

Thx~

zhenyuhe00 commented 11 months ago

Hi, we currently don't support beam search, since it requires some troublesome engineering efforts.