search

pytorch / PiPPy

Pipeline Parallelism for PyTorch
BSD 3-Clause "New" or "Revised" License
709 stars 86 forks source link

use torchDynamo instead of tracing to get graph? #434

Open Jack47 opened 2 years ago

Jack47 commented 2 years ago

hi guys, this project is awesome comparing with torch pipe, I have some models that are not supported by tracing. Would you guys have plan to support torchDynamo to get fx.graph?

torchDynamo creates this FX Graph through bytecode analysis and is designed to mix Python execution with compiled backends to get the best of both worlds: usability and performance.

anj-s commented 2 years ago

Yes, definitely. We do plan to support torchdynamo as the frontend in the near future.

Jack47 commented 2 years ago

Yes, definitely. We do plan to support torchdynamo as the frontend in the near future.

cool, glad to hear that, I will test it once released!

Vatshank commented 1 year ago

Hi @anj-s, is TorchDynamo support being worked on? I see dynamo mentioned in some of the code/comments under spmd/compiler and pippy/fx (and the pippy_dynamo.py example) but wasn't sure if the integration is a work in process or completed?

Happy to try contributing any missing pieces if you think that will be helpful!