search

StrongResearch / isc-demos

Deep learning examples for the Instant Super Computer
11 stars 0 forks source link

Distributed Model Parallel #40

Open adam-peaston-SC opened 10 months ago

adam-peaston-SC commented 10 months ago

Source / repo

https://pytorch.org/tutorials/intermediate/model_parallel_tutorial.html

[X] Distributed Model Parallel (DMP) [. ] Multi-Node DMP

Model description

[DESCRIPTION]

Dataset

[DATASET]

Literature benchmark source

[URL]

Literature benchmark performance

[DESCRIPTION] [VALUE/S]

Strong Compute result achieved

[VALUE/S]

Basic training config (as applicable)

Nodes: [N] Epochs: [N] Effective batch size: [N] Learning rate: [L] Optimizer: [OPT]

Logs gist

[URL]