training data

[HaoZhang990127]

Hi,

Thank you for your nice work.

You use video and image data in the training process. How much does adding image data help the final generated result? I found that adding image data did not improve the quality of the final generation. Any experience you can share about the training process?

Thank you so much.

[LinB203]

In fact, when we did the next version of the training, we found that the joint training didn't work very well. The image data is very important in providing the pre-training weights. But once the video can be generated, the image data is not important.

[HaoZhang990127]

Thank you for your quick reply.

Does "the image data is very important in providing the pre-training weights." means the image data plays a critical role in stage one?

[LinB203]

Unrelated to version 1.1, what I mean is that we need to train a weight with the image as an initialization of the video weights. For example we can retrain a bigger pixart, which will be loaded into the video model.

We will cover these details in the next version.

[HaoZhang990127]

Cool, thank you for your sharing