search

Alpha-VLLM / Lumina-T2X

Lumina-T2X is a unified framework for Text to Any Modality Generation
MIT License
2.09k stars 88 forks source link

error in Next-DiT #44

Open HuiZhang0812 opened 6 months ago

HuiZhang0812 commented 6 months ago

When I run Next-DiT according to the readme, I get the following problem when loading the DiT model. How can I solve it? KeyError: 'NextDiT_2B_GQA_patch2'

image
PommesPeter commented 6 months ago

Hi @HuiZhang0812 ,

Please pull the lastest code of Lumina-T2X repo, we added the NextDiT_2B_GQA_patch2 model in models.

HuiZhang0812 commented 6 months ago

Hi @HuiZhang0812 ,

Please pull the lastest code of Lumina-T2X repo, we added the NextDiT_2B_GQA_patch2 model in models.

Thanks, the model was loaded correctly. But during the inference process, I encountered the following new problems: TypeError: NextDiT.forward_with_cfg() got an unexpected keyword argument 'ntk_factor'

image image

I located that it was caused here, so I changed the value of "ntk_scaling" in settings.yaml to false and it worked. Will this have a big impact on the generated results? If so, how do you suggest to solve this problem?

image
PommesPeter commented 5 months ago

this is a typo for yaml file, we will fix it.

PommesPeter commented 5 months ago

please pull the latest code to use cli infer.