search

bigcode-project / starcoder2

Home of StarCoder2!
Apache License 2.0
1.76k stars 158 forks source link

"lm_head.weight" not in the parameters of Starcoder2-3B and Starcoder2-7B (Huggingface version) #18

Open renhouxing opened 7 months ago

renhouxing commented 7 months ago

There is no lm_head.weight in the parameters of Starcoder2-3B and Starcoder2-7B. Is it because of tied embedding?

NouamaneTazi commented 6 months ago

Yes!

renhouxing commented 6 months ago

ok, thanks for your response!