search

Alpha-VLLM / Lumina-T2X

Lumina-T2X is a unified framework for Text to Any Modality Generation
MIT License
1.82k stars 74 forks source link

error in Next-DiT #44

Open HuiZhang0812 opened 1 month ago

HuiZhang0812 commented 1 month ago

When I run Next-DiT according to the readme, I get the following problem when loading the DiT model. How can I solve it? KeyError: 'NextDiT_2B_GQA_patch2'

image
PommesPeter commented 1 month ago

Hi @HuiZhang0812 ,

Please pull the lastest code of Lumina-T2X repo, we added the NextDiT_2B_GQA_patch2 model in models.

HuiZhang0812 commented 1 month ago

Hi @HuiZhang0812 ,

Please pull the lastest code of Lumina-T2X repo, we added the NextDiT_2B_GQA_patch2 model in models.

Thanks, the model was loaded correctly. But during the inference process, I encountered the following new problems: TypeError: NextDiT.forward_with_cfg() got an unexpected keyword argument 'ntk_factor'

image image

I located that it was caused here, so I changed the value of "ntk_scaling" in settings.yaml to false and it worked. Will this have a big impact on the generated results? If so, how do you suggest to solve this problem?

image
PommesPeter commented 1 week ago

this is a typo for yaml file, we will fix it.

PommesPeter commented 1 week ago

please pull the latest code to use cli infer.