Closed yxchng closed 1 month ago
It's #771 ... openai force overrides activaiton to quickgelu (which is less efficient and uses more memory than nn.GELU), need to manually force via argument to create to use afterwards (or use a model config with quick gelu)
@rwightman how to manually force via argument?
gives