FasterDecoding REST issues

FasterDecoding / REST

REST: Retrieval-Based Speculative Decoding, NAACL 2024

Apache License 2.0

176 stars 11 forks source link

issues

Newest

Newest Most commented Recently updated Oldest Least commented Least recently updated

Support DraftRetriever datastore read/write for large vocab sizes (i.e. llama3+) and REST inference for llama3

#24 scandukuri closed 18 hours ago
2
Support DraftRetriever datastore read/write for large tokenizers and vocabulary sizes (i.e. llama3+)

#23 scandukuri closed 2 days ago
3
Lama2 70B-chat is not supported?

#22 Chlience opened 1 month ago
2
Tensor parallelism

#21 lethean1 opened 1 month ago
0
about retrieve sequence length

#20 Siegfried-qgf opened 1 month ago
1
OSError: failed to fill whole buffer

#19 Siegfried-qgf opened 1 month ago
6
How to adjust the maximum token lengths when drafting?

#18 zomss opened 2 months ago
3
LLama3 8B is not supported

#17 liranringel opened 4 months ago
3
If I want to change REST code to support multi-batch inference, what needs to be changed?

#16 yangbohust opened 6 months ago
0
How to support the `repetition_penalty` parameter?

#15 yangbohust opened 6 months ago
0
Segmentation Fault when calling libsais_int

#14 julianmukaj opened 6 months ago
0
ValueError: draft_choices was not cut enough / draft_len should not exceed 65

#13 yangbohust opened 6 months ago
3
What are the meanings of each parameter of the reader.search() function? k\choices\long

#12 yangbohust closed 2 weeks ago
9
pyo3_runtime.PanicException: called `Result::unwrap()` on an `Err` value: RuntimeError(StackOverflow)

#11 yangbohust closed 6 months ago
1
Gracefully handles when draft_choices is not cut enough

#10 wangpatrick57 closed 7 months ago
0
codellama-7b working on GPUs with 24GB memory

#9 wangpatrick57 closed 7 months ago
0
Ngram build cli

#8 wangpatrick57 closed 7 months ago
0
Small QOL changes

#7 wangpatrick57 closed 7 months ago
2
Questions about past_key_values_data

#6 reflectionie closed 7 months ago
2
What is the acceptance procedure in REST?

#5 jivanph closed 8 months ago
3
Cannot install the .whl

#4 hasuoshenyun closed 8 months ago
2
Incompatibility issues with llama-7b model

#3 preminstrel closed 9 months ago
6
Could REST support beam search sampling policy?

#2 leiwen83 closed 7 months ago
1
DraftRetriever wheels file missing

#1 drdsgvo closed 1 year ago
2