search

FasterDecoding / REST

REST: Retrieval-Based Speculative Decoding, NAACL 2024
Apache License 2.0
176 stars 11 forks source link

Tensor parallelism #21

Open lethean1 opened 1 month ago

lethean1 commented 1 month ago

I want to use tensor parallel with REST, but I do not find the config to start the tensor parallel, can you give me an example?