search

PKU-YuanGroup / Open-Sora-Plan

This project aim to reproduce Sora (Open AI T2V model), we wish the open source community contribute to this project.
MIT License
11.41k stars 1.02k forks source link

Training of VVAE #329

Open GFENGG opened 3 months ago

GFENGG commented 3 months ago

Hello, what does the curve of discriminator's gradient norm look like during normal training? Why is my curve gradually rising? (other losses and metrics are normal)

iShot_2024-07-07_10 58 24
LinB203 commented 3 months ago

Sorry, we don't have a statistical gradient. Can you try to inference to see if the quality is getting better with training?

GFENGG commented 3 months ago

Sorry, we don't have a statistical gradient. Can you try to inference to see if the quality is getting better with training?

The quality of the video improves before the convergence of the total loss, while the quality remains unchanged after the convergence of the total loss.

I found that when using Video-VAE (trained on 17 frames video) encoding and decoding for longer videos ( such as 33 or 65 frames) in one time (without time blend), the output result will gradually blur. Do you know the reason for this phenomenon?

serend1p1ty commented 3 months ago

@GFENGG Hello, can you reproduce the VAE results successfully?