Instructions on how to set TP/PP - Githubissues

huggingface / optimum-nvidia

Apache License 2.0

887 stars 86 forks source link

Instructions on how to set TP/PP #102

Open fxmarty opened 6 months ago

fxmarty commented 6 months ago

https://github.com/huggingface/optimum-nvidia/blob/main/examples/text-generation.py is currently empty in that regard