Open sudhakarsingh27 opened 2 weeks ago
Expose rotary_base as an arg instead of hardcoding to 10000
rotary_base
Fixes # (issue) https://github.com/NVIDIA/TransformerEngine/issues/849
/te-ci pytorch
Description
Expose
rotary_base
as an arg instead of hardcoding to 10000Fixes # (issue) https://github.com/NVIDIA/TransformerEngine/issues/849
Type of change