No warningmessage when load models with a gamma or beta parameter

zly-idleness commented 2 months ago

System Info

transformers version: 4.44.2
Platform: Linux-5.4.0-135-generic-x86_64-with-glibc2.31
Python version: 3.10.14
Huggingface_hub version: 0.24.6
Safetensors version: 0.4.4
Accelerate version: 0.33.0
Accelerate config: not found
PyTorch version (GPU?): 2.2.0 (True)
Tensorflow version (GPU?): not installed (NA)
Flax version (CPU?/GPU?/TPU?): not installed (NA)
Jax version: not installed
JaxLib version: not installed
Using distributed or parallel set-up in script?:
Using GPU in script?:
GPU type: NVIDIA A100-SXM4-80GB

Who can help?

No response

Information

[ ] The official example scripts
[X] My own modified scripts

Tasks

[X] An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
[X] My own task or dataset (give details below)

Reproduction

Just like #29554 previous commit just info at load_state and no warning message if you load model just in

4061 def _load_pretrained_model(
            ...
    ):

there will be no warning message at all !!

Expected behavior

(vqa-audio) (base) jeeves@notebook-5064-cadence:~/ChatTTS/rhapsodyaudio$ python tmp_save_pretrain.py 
bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)
Loading checkpoint shards: 100%|██████████████████████████████████████████████████████████████████████████████| 8/8 [00:16<00:00,  2.02s/it]
Some weights of Qwen2AudioForConditionalChatTTS were not initialized from the model checkpoint at /mnt/data/user/tc_agi/luoyuanZ/ChatTTS_default and are newly initialized: 

['tts.dvae.decoder.decoder_block.0.gamma', 'tts.dvae.decoder.decoder_block.1.gamma', 'tts.dvae.decoder.decoder_block.10.gamma', 'tts.dvae.decoder.decoder_block.11.gamma', 'tts.dvae.decoder.decoder_block.2.gamma', 'tts.dvae.decoder.decoder_block.3.gamma', 'tts.dvae.decoder.decoder_block.4.gamma', 'tts.dvae.decoder.decoder_block.5.gamma', 'tts.dvae.decoder.decoder_block.6.gamma', 'tts.dvae.decoder.decoder_block.7.gamma', 'tts.dvae.decoder.decoder_block.8.gamma', 'tts.dvae.decoder.decoder_block.9.gamma', 'tts.dvae.encoder.decoder_block.0.gamma', 'tts.dvae.encoder.decoder_block.1.gamma', 'tts.dvae.encoder.decoder_block.10.gamma', 'tts.dvae.encoder.decoder_block.11.gamma', 'tts.dvae.encoder.decoder_block.2.gamma', 'tts.dvae.encoder.decoder_block.3.gamma', 'tts.dvae.encoder.decoder_block.4.gamma', 'tts.dvae.encoder.decoder_block.5.gamma', 'tts.dvae.encoder.decoder_block.6.gamma', 'tts.dvae.encoder.decoder_block.7.gamma', 'tts.dvae.encoder.decoder_block.8.gamma', 'tts.dvae.encoder.decoder_block.9.gamma']

LysandreJik commented 2 months ago

cc @amyeroberts and @muellerz as you have been working on a PR for that

amyeroberts commented 2 months ago

Hi @zly-idleness, thanks for raising!

We merged in #32656 which should address this. It's currently just on main and will be part of the next release. If you want to try the development branch with this feature, you can install from source: pip install git+https://github.com/huggingface/transformers

zly-idleness commented 2 months ago

Hi @zly-idleness, thanks for raising!

We merged in #32656 which should address this. It's currently just on main and will be part of the next release. If you want to try the development branch with this feature, you can install from source: pip install git+https://github.com/huggingface/transformers

I have reviewed the recent commit, but it failed to solve the issue I'm facing. There have been prior commits addressing this bug, and my current version of the code already includes mechanisms to detect such behavior. However, the expected warning messages from the logger are not being outputted. To address this, I've submitted a pull request that enhances the _load_pretrained_model() function with additional warnings and informational messages. With these changes, the warning messages are now correctly displayed.

amyeroberts commented 2 months ago

Ah, OK, I see - thanks for explaining and opening up a PR to address!

github-actions[bot] commented 1 month ago

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

huggingface / transformers