Tensor Parallelism - Githubissues

dilab-zju / self-speculative-decoding

Code associated with the paper **Draft & Verify: Lossless Large Language Model Acceleration via Self-Speculative Decoding**

Apache License 2.0

141 stars 9 forks source link

Open lethean1 opened 1 month ago

lethean1 commented 1 month ago

I want to use tensor parallelism with your work, but I do not find the config to start the tensor parallel, can you give me an example?

junzhang-zj commented 1 month ago

Our code cannot be adapted to TP, and we welcome the community to contribute a more general version.