ValueError: unknown mid_block_type : UNetMidBlock2DCrossAttn on model trained with dreambooth

[ypsl]

Hello, I've tried to train the model that I've got with diffuser's dreambooth example: https://github.com/huggingface/diffusers/blob/main/examples/dreambooth/train_dreambooth.py but stuck with: Traceback (most recent call last): File "/home/ubuntu/Tune-A-Video/train_tuneavideo.py", line 367, in main(OmegaConf.load(args.config)) File "/home/ubuntu/Tune-A-Video/train_tuneavideo.py", line 109, in main unet = UNet3DConditionModel.from_pretrained_2d(pretrained_model_path, subfolder="unet") File "/home/ubuntu/Tune-A-Video/tuneavideo/models/unet.py", line 440, in from_pretrained_2d model = cls.from_config(config) File "/home/ubuntu/Tune-A-Video/venv/lib/python3.9/site-packages/diffusers/configuration_utils.py", line 210, in from_config model = cls(init_dict) File "/home/ubuntu/Tune-A-Video/venv/lib/python3.9/site-packages/diffusers/configuration_utils.py", line 567, in inner_init init(self, *args, **init_kwargs) File "/home/ubuntu/Tune-A-Video/tuneavideo/models/unet.py", line 158, in init raise ValueError(f"unknown mid_block_type : {mid_block_type}") ValueError: unknown mid_block_type : UNetMidBlock2DCrossAttn Will it be supported? Maybe you have some tips on how to implement this Thanks

[zhangjiewu]

https://github.com/showlab/Tune-A-Video/blob/85b7c50c4f6aa1f35470e1a92142e02f81ddfed0/tuneavideo/models/unet.py#L426-L431

try adding config["mid_block_type"] = "UNetMidBlock3DCrossAttn" after L431.

[ypsl]

Thank you! It helped to continue the training. Not sure if it worked correctly, the video variable in inference.py now contains empty tensors(= empty gif file).

[ypsl]

Also tried to train on https://huggingface.co/andite/anything-v4.0 and got the same error

[zhangjiewu]

can you try this bear-guitar video?

[ypsl]

Reinstalling the dependencies helped me. Thank you for the attention!