Closed sogartar closed 3 weeks ago
Sharding specs are concise description of tensor and layer sharding. They can be applied to tensors and thetas to get the sharded counterpart.
This is mostly the pluming around specific cases. Only sharing of the channel dimension in group normalization is provided.
Sharding specs are concise description of tensor and layer sharding. They can be applied to tensors and thetas to get the sharded counterpart.
This is mostly the pluming around specific cases. Only sharing of the channel dimension in group normalization is provided.