FasterDecoding / REST

REST: Retrieval-Based Speculative Decoding, NAACL 2024
Apache License 2.0
166 stars 10 forks source link

about retrieve sequence length #20

Open Siegfried-qgf opened 2 weeks ago

Siegfried-qgf commented 2 weeks ago

I followed your suggestion to modify retriever to adapt to Qwen2.5 When I run function "search". The return sequence seems too short. Though I change the parameter "long", It doesn't work.