Closed piddnad closed 4 months ago
I'm wondering, was this done by loading the PixArt-alpha pretrained weights from the High aesthetics stage, and then using to the 33M internal-sigma data for adaption? Is my understanding correct?
This is correct. Share some results of your training here.
Here are some of my results. The resulting images seem to have some chunking and blurring:
prompt: A lovely young lady, with a smile on her face...
prompt: city skyline at night...
which training and test code are you using?
which training and test code are you using?
I'm using the training and validation code based on train_diffusers.py of PixArt-alpha.
Check your vae and scale_factor. BTW, the training with diffusers is not stable. That's why we haven't changed all the code base to diffusres.
Hello, I have carefully reviewed the vae code and used the original training code, and the results have not changed.
However, I have conducted a few more adaption experiments with the SDXL VAE, and there are some interesting findings to share.
The setups of the experiments were as follows:
Through these experiments, I observed some interesting phenomena: in terms of the final generated effect after the transfer, 4 > 3 ≈ 2 >> 1. Therefore, I speculate that both high-aesthetic pre-trained models and high-aesthetic data are beneficial to the VAE adaption.
Below are some visual examples: (from left to right: 4,2,3,1)
Cool. Pretty interesting results. Thanks a lot for sharing.
Thank you for sharing such impressive work!
I am particularly interested in the VAE adaption stage mentioned in the paper. It's mentioned that it was conducted at a 256x256 resolution.
I'm wondering, was this done by loading the PixArt-alpha pretrained weights from the High aesthetics stage, and then using to the 33M internal-sigma data for adaption? Is my understanding correct?
I have tried a training using PixArt-alpha 256-SAM weights, replacing SD1.5 VAE with SDXL VAE, training on SAM data and it seems to be difficult to converging in the short term (for now 10k steps), do you know what the problem might be?
Thank you in advance for your response.