Vchitect / Latte

Latte: Latent Diffusion Transformer for Video Generation.
Apache License 2.0
1.45k stars 147 forks source link

Cannot find model:LatteT2V.from_pretrained_2d #31

Closed bosima closed 4 months ago

bosima commented 4 months ago
~/Latte# bash sample/t2v.sh
/root/miniconda3/envs/latte/lib/python3.12/site-packages/diffusers/utils/outputs.py:63: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead.
  torch.utils._pytree._register_pytree_node(
/root/miniconda3/envs/latte/lib/python3.12/site-packages/diffusers/utils/outputs.py:63: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead.
  torch.utils._pytree._register_pytree_node(
Traceback (most recent call last):
  File "/root/Latte/sample/sample_t2v.py", line 160, in <module>
    main(OmegaConf.load(args.config))
  File "/root/Latte/sample/sample_t2v.py", line 30, in main
    transformer_model = get_models(args).to(device, dtype=torch.float16)
                        ^^^^^^^^^^^^^^^^
  File "/root/Latte/models/__init__.py", line 42, in get_models
    return LatteT2V.from_pretrained_2d(pretrained_model_path, subfolder="transformer")
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/Latte/models/latte_t2v.py", line 992, in from_pretrained_2d
    raise RuntimeError(f"{model_file} does not exist")
RuntimeError: None does not exist

Code:

        model = cls.from_config(config)

        model_files = [
            os.path.join(pretrained_model_path, 'diffusion_pytorch_model.bin'),
            os.path.join(pretrained_model_path, 'diffusion_pytorch_model.safetensors')
        ]

        model_file = None

        for fp in model_files:
            if os.path.exists(fp):
                model_file = fp

        if not model_file:
            raise RuntimeError(f"{model_file} does not exist")

the transformer does not have these two models:

image
bosima commented 4 months ago

The local code is outdated.