Some weights of the model checkpoint at ./pretrained_model/wav2vec2-base-960h were not used when initializing Wav2Vec2Model: ['lm_head.weight', 'lm_head.bias']

[dongyiqi]

call python -m scripts.audio2vid --config ./configs/prompts/animation_audio.yaml -W 512 -H 512

error: Some weights of the model checkpoint at ./pretrained_model/wav2vec2-base-960h were not used when initializing Wav2Vec2Model: ['lm_head.weight', 'lm_head.bias']

• This IS expected if you are initializing Wav2Vec2Model from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
• This IS NOT expected if you are initializing Wav2Vec2Model from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model). Some weights of Wav2Vec2Model were not initialized from the model checkpoint at ./pretrained_model/wav2vec2-base-960h and are newly initialized: ['wav2vec2.masked_spec_embed'] You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference. Some weights of the model checkpoint were not used when initializing UNet2DConditionModel: ['conv_norm_out.weight, conv_norm_out.bias, conv_out.weight, conv_out.bias'] C:\ProgramData\anaconda3\envs\AniPortrait-Env\lib\site-packages\torch_utils.py:776: UserWarning: TypedStorage is deprecated. It will be removed in the future and UntypedStorage will be the only storage class. This should only matter to you if you are using storages directly. To access UntypedStorage directly, use tensor.untyped_storage() instead of tensor.storage() return self.fget.get(instance, owner)() Traceback (most recent call last): File "C:\ProgramData\anaconda3\envs\AniPortrait-Env\lib\runpy.py", line 196, in _run_module_as_main return _run_code(code, main_globals, None, File "C:\ProgramData\anaconda3\envs\AniPortrait-Env\lib\runpy.py", line 86, in _run_code exec(code, run_globals) File "D:\Dyq\AniPortrait\scripts\audio2vid.py", line 225, in main() File "D:\Dyq\AniPortrait\scripts\audio2vid.py", line 87, in main image_enc = CLIPVisionModelWithProjection.from_pretrained( File "C:\ProgramData\anaconda3\envs\AniPortrait-Env\lib\site-packages\transformers\modeling_utils.py", line 2305, in from_pretrained config, model_kwargs = cls.config_class.from_pretrained( File "C:\ProgramData\anaconda3\envs\AniPortrait-Env\lib\site-packages\transformers\models\clip\configuration_clip.py", line 233, in from_pretrained config_dict, kwargs = cls.get_config_dict(pretrained_model_name_or_path, kwargs) File "C:\ProgramData\anaconda3\envs\AniPortrait-Env\lib\site-packages\transformers\configuration_utils.py", line 574, in get_config_dict config_dict, kwargs = cls._get_config_dict(pretrained_model_name_or_path, kwargs) File "C:\ProgramData\anaconda3\envs\AniPortrait-Env\lib\site-packages\transformers\configuration_utils.py", line 629, in _get_config_dict resolved_config_file = cached_file( File "C:\ProgramData\anaconda3\envs\AniPortrait-Env\lib\site-packages\transformers\utils\hub.py", line 388, in cached_file raise EnvironmentError( OSError: ./pretrained_model/image_encoder does not appear to have a file named config.json. Checkout 'https://huggingface.co/./pretrained_model/image_encoder/None' for available files.

[steeveen]

Have you fixed this?

[motivated3]

Have you fixed this?