Closed wangyuwen1999 closed 3 months ago
I want to use tensor parallel with lookhead, but I do not find the config to start the tensor parallel, can you give me an example?
Tensor parallel is not compatible with lookhead yet. You can choose only to use tensor parallel (and no lookahead). L28-L35 in your figure is an example if you set GPUS larger.
I want to use tensor parallel with lookhead, but I do not find the config to start the tensor parallel, can you give me an example?