Closed cyanguwa closed 2 weeks ago
Add appropriate logic for tp_group_initialized after making DotProductAttention a TransformerEngineBaseModule.
tp_group_initialized
DotProductAttention
TransformerEngineBaseModule
Please list the changes introduced in this PR:
DotProductAttention.__init__()
/te-ci pytorch
Description
Add appropriate logic for
tp_group_initialized
after makingDotProductAttention
aTransformerEngineBaseModule
.Type of change
Changes
Please list the changes introduced in this PR:
tp_group_initialized
inDotProductAttention.__init__()
Checklist: