search

FasterDecoding / REST

REST: Retrieval-Based Speculative Decoding, NAACL 2024
Apache License 2.0
166 stars 10 forks source link

Tensor parallelism #21

Open lethean1 opened 5 days ago

lethean1 commented 5 days ago

I want to use tensor parallel with REST, but I do not find the config to start the tensor parallel, can you give me an example?