search

pytorch / tutorials

PyTorch tutorials.
https://pytorch.org/tutorials/
BSD 3-Clause "New" or "Revised" License
8.13k stars 4.04k forks source link

[BUG] - missing file - tp_tutorial.py #2956

Closed SeungjunNah closed 1 month ago

SeungjunNah commented 3 months ago

Add Link

https://github.com/pytorch/tutorials/blob/main/intermediate_source/TP_tutorial.rst#how-to-apply-tensor-parallel

Describe the bug

Hi, in this tutorial, there is an example command suggested to run the tutorial code. However, as of today, tp_tutorial.py file does not exist in this repo. It looks like the file was accidentally not pushed. It would be great if it could be added.

# run this via torchrun: torchrun --standalone --nproc_per_node=8 ./tp_tutorial.py

Describe your environment

Linux - Ubuntu.

svekars commented 3 months ago

cc: @wanchaol @tianyu-l

tianyu-l commented 1 month ago

Sorry this comment was not intended to be put here.

I've updated the tutorial to remove this comment https://github.com/pytorch/tutorials/pull/2998

To see/run the end-to-end code example, please follow the link in the tutorial to Tensor Parallel examples in the pytorch/examples repository