search

microsoft / torchscale

Foundation Architecture for (M)LLMs
https://aka.ms/GeneralAI
MIT License
2.98k stars 201 forks source link

Could you please explain the reason behind defining TEMPERATURE_FOR_L_UAX in the code without actually using it? #63

Closed Ruiyuan-Zhang closed 11 months ago

Ruiyuan-Zhang commented 11 months ago

As the title says, there was only one result.

image
donglixp commented 11 months ago

https://github.com/xy980523/MoEc_model/blob/45c83c8c483eb278d334feded844e0d3d5d96e27/unilm/modules/tmoe/top1gate.py#L86