Open HaoZhang990127 opened 1 month ago
In fact, when we did the next version of the training, we found that the joint training didn't work very well. The image data is very important in providing the pre-training weights. But once the video can be generated, the image data is not important.
Thank you for your quick reply.
Does "the image data is very important in providing the pre-training weights." means the image data plays a critical role in stage one?
Unrelated to version 1.1, what I mean is that we need to train a weight with the image as an initialization of the video weights. For example we can retrain a bigger pixart, which will be loaded into the video model.
We will cover these details in the next version.
Cool, thank you for your sharing
Hi,
Thank you for your nice work.
You use video and image data in the training process. How much does adding image data help the final generated result? I found that adding image data did not improve the quality of the final generation. Any experience you can share about the training process?
Thank you so much.