Change RotaryEmbedding to GPTNeoXRotaryEmbedding for Transformers 4.31.0

togethercomputer / OpenChatKit

Apache License 2.0

9.01k stars 1.01k forks source link

Change RotaryEmbedding to GPTNeoXRotaryEmbedding for Transformers 4.31.0 #168

Closed azahed98 closed 1 year ago

azahed98 commented 1 year ago

Addresses Issue #167, caused by renamed classes in Transformers 4.31.0.

This pull request updates the required RotaryEmbedding class for GPTNeoX to GPTNeoXRotaryEmbeedding within modules/hf_gptneox_modules.py to update to transformers==4.31.0.