search

FreedomIntelligence / AceGPT

Apache License 2.0
111 stars 7 forks source link

some layers are not loaded #3

Closed Mahmoud-SaadEl-Din closed 6 months ago

Mahmoud-SaadEl-Din commented 11 months ago

Why do i recieve this msg when running "python web_demo.py"? ""Some weights of LlamaForCausalLM were not initialized from the model checkpoint at FreedomIntelligence/AceGPT-7B-chat and are newly initialized: ['model.layers.21.self_attn.rotary_emb.inv_freq', 'model.layers.8.self_attn.rotary_emb.inv_freq', 'model.layers.18.self_attn.rotary_emb.inv_freq', 'model.layers.29.self_attn.rotary_emb.inv_freq', 'model.layers.1.self_attn.rotary_emb.inv_freq', 'model.layers.10.self_attn.rotary_emb.inv_freq', 'model.layers.15.self_attn.rotary_emb.inv_freq', 'model.layers.14.self_attn.rotary_emb.inv_freq', 'model.layers.11.self_attn.rotary_emb.inv_freq', 'model.layers.3.self_attn.rotary_emb.inv_freq', 'model.layers.17.self_attn.rotary_emb.inv_freq', 'model.layers.6.self_attn.rotary_emb.inv_freq', 'model.layers.7.self_attn.rotary_emb.inv_freq', 'model.layers.19.self_attn.rotary_emb.inv_freq', 'model.layers.20.self_attn.rotary_emb.inv_freq', 'model.layers.28.self_attn.rotary_emb.inv_freq', 'model.layers.31.self_attn.rotary_emb.inv_freq', 'model.layers.16.self_attn.rotary_emb.inv_freq', 'model.layers.25.self_attn.rotary_emb.inv_freq', 'model.layers.0.self_attn.rotary_emb.inv_freq', 'model.layers.2.self_attn.rotary_emb.inv_freq', 'model.layers.4.self_attn.rotary_emb.inv_freq', 'model.layers.9.self_attn.rotary_emb.inv_freq', 'model.layers.22.self_attn.rotary_emb.inv_freq', 'model.layers.27.self_attn.rotary_emb.inv_freq', 'model.layers.30.self_attn.rotary_emb.inv_freq', 'model.layers.26.self_attn.rotary_emb.inv_freq', 'model.layers.5.self_attn.rotary_emb.inv_freq', 'model.layers.13.self_attn.rotary_emb.inv_freq', 'model.layers.23.self_attn.rotary_emb.inv_freq', 'model.layers.24.self_attn.rotary_emb.inv_freq', 'model.layers.12.self_attn.rotary_emb.inv_freq'] You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.

jianqing666 commented 10 months ago

Hi， we have recently updated our model on Hugging Face. This update addresses the problem you're experiencing. To resolve the issue, please download the latest version of the model from Hugging Face and test it again.

hhwer commented 6 months ago

@Mahmoud-SaadEl-Din "self_attn.rotary_emb.inv_freq" is not trainable weights and I guess re-initialize will not inluence it's value(https://github.com/LianjiaTech/BELLE/issues/349#issuecomment-1544942632). And this warning is caused by transformers version.