i2v 通道错误

[O-O1024]

如果只设置起始图片，是可以生成视频的。但是加上结尾图片，就出错了，图片是 png 格式

Traceback (most recent call last):
  File "/root/EasyAnimate/predict_i2v.py", line 247, in <module>
    sample = pipeline(
  File "/root/miniconda3/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/root/EasyAnimate/easyanimate/pipeline/pipeline_easyanimate_inpaint.py", line 922, in __call__
    latents_outputs = self.prepare_latents(
  File "/root/EasyAnimate/easyanimate/pipeline/pipeline_easyanimate_inpaint.py", line 535, in prepare_latents
    video_bs = self.vae.encode(video_bs)[0]
  File "/root/miniconda3/lib/python3.10/site-packages/diffusers/utils/accelerate_utils.py", line 46, in wrapper
    return method(self, *args, **kwargs)
  File "/root/EasyAnimate/easyanimate/models/autoencoder_magvit.py", line 257, in encode
    return self.tiled_encode(x, return_dict=return_dict)
  File "/root/EasyAnimate/easyanimate/models/autoencoder_magvit.py", line 350, in tiled_encode
    tile = self.encoder(tile)
  File "/root/miniconda3/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/root/miniconda3/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1541, in _call_impl
    return forward_call(*args, **kwargs)
  File "/root/EasyAnimate/easyanimate/vae/ldm/models/omnigen_enc_dec.py", line 194, in forward
    next_frames = self.single_forward(x[:, :, i: i + self.mini_batch_encoder, :, :], previous_features, after_features)
  File "/root/EasyAnimate/easyanimate/vae/ldm/models/omnigen_enc_dec.py", line 157, in single_forward
    x = self.conv_in(x)
  File "/root/miniconda3/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/root/miniconda3/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1541, in _call_impl
    return forward_call(*args, **kwargs)
  File "/root/EasyAnimate/easyanimate/vae/ldm/modules/vaemodules/common.py", line 81, in forward
    return super().forward(x)
  File "/root/miniconda3/lib/python3.10/site-packages/torch/nn/modules/conv.py", line 610, in forward
    return self._conv_forward(input, self.weight, self.bias)
  File "/root/miniconda3/lib/python3.10/site-packages/torch/nn/modules/conv.py", line 605, in _conv_forward
    return F.conv3d(
RuntimeError: Given groups=1, weight of size [128, 3, 3, 3, 3], expected input[1, 4, 10, 384, 384] to have 3 channels, but got 4 channels instead

[O-O1024]

另外文生图也出错了

/root/miniconda3/lib/python3.10/site-packages/diffusers/models/transformers/transformer_2d.py:34: FutureWarning: `Transformer2DModelOutput` is deprecated and will be removed in version 1.0.0. Importing `Transformer2DModelOutput` from `diffusers.models.transformer_2d` is deprecated and this will be removed in a future version. Please use `from diffusers.models.modeling_outputs import Transformer2DModelOutput`, instead.
  deprecate("Transformer2DModelOutput", "1.0.0", deprecation_message)
model_name= models/Diffusion_Transformer/EasyAnimateV3-XL-2-InP-512x512
loaded 2D transformer's pretrained weights from models/Diffusion_Transformer/EasyAnimateV3-XL-2-InP-512x512/transformer ...
The config attributes {'casual_3d': False, 'casual_3d_upsampler_index': None, 'decay': 0.9999, 'enable_uvit': True, 'fake_3d': False, 'inv_gamma': 1.0, 'min_decay': 0.0, 'motion_module_kwargs_even': {'attention_block_types': ['Temporal_Self', 'Temporal_Self'], 'block_size': 1, 'num_attention_heads': 16, 'num_transformer_block': 1, 'temporal_attention_dim_div': 1, 'temporal_position_encoding': True, 'temporal_position_encoding_max_len': 4096}, 'motion_module_kwargs_odd': {'attention_block_types': ['Temporal_Self', 'Global_Self'], 'block_size': 1, 'num_attention_heads': 16, 'num_transformer_block': 1, 'temporal_attention_dim_div': 1, 'temporal_position_encoding': True, 'temporal_position_encoding_max_len': 4096}, 'motion_module_type': 'Vanilla', 'optimization_step': 4400, 'patch_3d': False, 'power': 0.6666666666666666, 'time_patch_size': None, 'time_position_encoding_before_transformer': False, 'update_after_step': 0, 'use_ema_warmup': False} were passed to Transformer2DModel, but are not expected and will be ignored. Please verify your config.json configuration file.

Traceback (most recent call last):
  File "/root/EasyAnimate/predict_t2i.py", line 57, in <module>
    transformer = Transformer2DModel.from_pretrained(
  File "/root/EasyAnimate/easyanimate/models/transformer2d.py", line 518, in from_pretrained
    model = cls.from_config(config, **transformer_additional_kwargs)
  File "/root/miniconda3/lib/python3.10/site-packages/diffusers/configuration_utils.py", line 260, in from_config
    model = cls(**init_dict)
  File "/root/miniconda3/lib/python3.10/site-packages/diffusers/configuration_utils.py", line 658, in inner_init
    init(self, *args, **init_kwargs)
  File "/root/EasyAnimate/easyanimate/models/transformer2d.py", line 199, in __init__
    basic_block = {
KeyError: 'global_motionmodule'

[yunkchen]

另外文生图也出错了

Traceback (most recent call last):
  File "/root/EasyAnimate/predict_t2i.py", line 57, in <module>
    transformer = Transformer2DModel.from_pretrained(
  File "/root/EasyAnimate/easyanimate/models/transformer2d.py", line 518, in from_pretrained
    model = cls.from_config(config, **transformer_additional_kwargs)
  File "/root/miniconda3/lib/python3.10/site-packages/diffusers/configuration_utils.py", line 260, in from_config
    model = cls(**init_dict)
  File "/root/miniconda3/lib/python3.10/site-packages/diffusers/configuration_utils.py", line 658, in inner_init
    init(self, *args, **init_kwargs)
  File "/root/EasyAnimate/easyanimate/models/transformer2d.py", line 199, in __init__
    basic_block = {
KeyError: 'global_motionmodule'

是不是没有载入模型

[O-O1024]

The config attributes

在这里出错了：

transformer = Transformer2DModel.from_pretrained(
    model_name, 
    subfolder="transformer"
).to(weight_dtype)

t2i, t2v, i2v 运行时都有有一句 The config attributes {..................} were passed to Transformer2DModel, but are not expected and will be ignored. Please verify your config.json configuration file.

不知道是不是模型的问题，我用的是 https://huggingface.co/Kijai/EasyAnimate-bf16，

第一个问题是图片格式导致的吗？

[yunkchen]

The config attributes

在这里出错了：

transformer = Transformer2DModel.from_pretrained(
    model_name, 
    subfolder="transformer"
).to(weight_dtype)

t2i, t2v, i2v 运行时都有有一句 The config attributes {..................} were passed to Transformer2DModel, but are not expected and will be ignored. Please verify your config.json configuration file.

不知道是不是模型的问题，我用的是 https://huggingface.co/Kijai/EasyAnimate-bf16，

第一个问题是图片格式导致的吗？

你知道的，是模型的问题，下面才是我们模型在hf上的地址

https://huggingface.co/alibaba-pai/EasyAnimateV3-XL-2-InP-768x768