Some more information regarding parameters please

[FurkanGozukara]

Currently I am recording a tutorial video to animation with stable diffusion

I have got excellent quality consistency

Here example video

https://user-images.githubusercontent.com/19240467/227747006-2f09449b-a046-45ad-b0c4-e3a8d7bb718c.mp4

I will also upscale this to 1024x1024

Now I want to reduce flickering problem with your AI

could you give me some optimal parameters for such thing?

Lets say I have 1024x1024 resolution and 2000 frames

So many parameters. So how should I modify them? making iters_num = 100k should I also change samples_batch rgb_coeff alpha_bootstrapping_factor alpha_flow_factor stop_bootstrapping_iteration sparsity_coeff

Could you give some idea about which one of these should be changed? thank you so much

{ "results_folder_name": "results", "maximum_number_of_frames": 2000, "resx": 1024, "resy": 1024, "iters_num": 100001, "samples_batch": 10000, "optical_flow_coeff": 500.0, "evaluate_every": 10000, "derivative_amount": 1, "rgb_coeff": 5000, "rigidity_coeff": 1.0, "uv_mapping_scale": 0.8, "pretrain_mapping1": true, "pretrain_mapping2": true, "alpha_bootstrapping_factor": 2000.0, "alpha_flow_factor": 4900.0, "positional_encoding_num_alpha": 5, "number_of_channels_atlas": 256, "number_of_layers_atlas": 8, "number_of_channels_alpha": 256, "number_of_layers_alpha": 8, "stop_bootstrapping_iteration": 10000, "number_of_channels_mapping1": 256, "number_of_layers_mapping1": 6, "number_of_channels_mapping2": 256, "number_of_layers_mapping2": 4, "gradient_loss_coeff": 1000, "use_gradient_loss": true, "sparsity_coeff": 1000.0, "positional_encoding_num_atlas": 10, "use_positional_encoding_mapping1": false, "number_of_positional_encoding_mapping1": 4, "use_positional_encoding_mapping2": false, "number_of_positional_encoding_mapping2": 2, "pretrain_iter_number": 100, "load_checkpoint": false, "checkpoint_path": "", "include_global_rigidity_loss": true, "global_rigidity_derivative_amount_fg": 100, "global_rigidity_derivative_amount_bg": 100, "global_rigidity_coeff_fg": 5.0, "global_rigidity_coeff_bg": 50.0, "stop_global_rigidity": 5000, "add_to_experiment_folder_name": "" }

@xrenaa @ChenyangLEI

[xrenaa]

Hi, I think our method can perfectly work on your video. There are somethings you can try:

1. Considering your video is 36s, maybe you can first try fps=10 to have 360 frame and set the iters_num to 100001.
2. You can change samples_batch which is kind of bacth-size according to your GPU mermoy.
3. For rgb_coeff alpha_bootstrapping_factor alpha_flow_factor stop_bootstrapping_iteration sparsity_coeff, you can leave if unchanged.
4. For the optical flow loss, we only use optical_flow_coeff in current setting.
5. We wil realease the segmentation support very soon which can make the video with main object better. Please refer to https://github.com/ChenyangLEI/All-In-One-Deflicker/issues/8 for a demo.

[FurkanGozukara]

Hi, I think our method can perfectly work on your video. There are somethings you can try:

1. Considering your video is 36s, maybe you can first try fps=10 to have 360 frame and set the iters_num to 100001.
2. You can change samples_batch which is kind of bacth-size according to your GPU mermoy.
3. For rgb_coeff alpha_bootstrapping_factor alpha_flow_factor stop_bootstrapping_iteration sparsity_coeff, you can leave if unchanged.
4. For the optical flow loss, we only use optical_flow_coeff in current setting.
5. We wil realease the segmentation support very soon which can make the video with main object better. Please refer to Applying to MMD to 2D output by ControlNet #8 for a demo.

thank you for reply

I used default settings on a 43 second video and quality is too degraded any ideas? I need to keep FPS and the picture quality same

are we limited with 10 fps what is the logic?

default settings All-In-One-Deflicker processed

https://user-images.githubusercontent.com/19240467/227769437-aa752f64-eca9-4cdc-8c84-8aa94b6b9d6b.mp4

original file

https://user-images.githubusercontent.com/19240467/227769467-d7558514-629d-4fa0-8f49-f93c5a113672.mp4

[ChenyangLEI]

I think you need to keep the fps as the same with the original video. Now the FPS is obvious lower than your orignal file. Do you know the original FPS?

[FurkanGozukara]

I think you need to keep the fps as the same with the original video. Now the FPS is obvious lower than your orignal file. Do you know the original FPS?

original fps is 30 how do i keep fps same? i dont see it in your configuration file

[ChenyangLEI]

python test.py --fps 30

[FurkanGozukara]

python test.py --fps 30

i will do

can we keep up the quality as well? if so how?

this is the tutorial video I made however I wasnt able to get good results with yours so i didnt used it

26.) Automatic1111 Web UI - PC - Free Video To Anime - Generate An EPIC Animation From Your Phone Recording By Using Stable Diffusion AI

[xrenaa]

Hi @FurkanGozukara,

I think you can change:

1. optical_flow_coeff to 5 or 50. When this flow loss weight is larger, the video is smooth but the results will be degraded.
2. iters_num to 400000 to make the video convergence. (And set evaluate_every to 50000 to make the training faster)

[xrenaa]

Since there are no update, we close this issue. We update the segmentation version. You are welcome to try it. I think it is helpful for your case.

[awt135]

good, this is what i want find , i used stable diffuion aslo have this question ,thx

[gvonkreisler]

If there is any question of quality in this project: DO NOT CHANGE THE RESOLUTION OF THE INPUT. That is a no go for any one with a little bit of knowledge. Bikubic,bla,bla, something all this is crap. If, like me comes from TV or Film nothing is so bad. Make some version for black and white( First layer one channel) and one for the color version wit [3,x,x]. This will save gpu memory and most of the flickering video is BW. You will say: All the video we get are the small ones from the trainings sets all ready to small to make difference. WRONG i am sitting on Old Color films from 1938 to the 60s.Most of the public film librarys have millions on meters of old films for training.