search

apapiu / transformer_latent_diffusion

Text to Image Latent Diffusion using a Transformer core
MIT License
141 stars 17 forks source link

Weird image when running the sample generator #19

Open metatl opened 6 months ago

metatl commented 6 months ago

Just installed and give it a test of the trained model.

from tld.configs import LTDConfig, DenoiserConfig, TrainConfig from tld.diffusion import DiffusionTransformer

denoiser_cfg = DenoiserConfig(n_channels=4) #configure your model here. cfg = LTDConfig(denoiser_cfg=denoiser_cfg)

diffusion_transformer = DiffusionTransformer(cfg)

out = diffusion_transformer.generate_image_from_text(prompt="a cute cat") I got something like this: image

Not sure why.

apapiu commented 6 months ago

Hey @metatl, try using the legacy_dh_order branch - the model was trained with a small but annoying difference in ordering of the hidden dimensions and head dimensions and unfortunately I haven't found a simple way to fix that. Let me know if that doesn't work.