sky timelapse preprocessing

[Jaeyun-Song]

Hi, thank you for your great jobs and code release!

I have a trouble in training models on sky timelapse. Among sky timelapse videos, there are very short videos which have less then 256 frames, so the command you recommended does not work.

In detail, I downloaded dataset from this link. And then I preprocessed the dataset with crop_video_dataset.py along with satisfying suggested dataset structure. I run "python src/infra/launch.py hydra.run.dir=. exp_suffix=my_experiment_name env=local dataset=sky_timelapse dataset.resolution=256 num_gpus=4 model.generator.time_enc.min_period_len=256". I encountered "IndexError: list index out of range" in layers.py", line 396, in random_frame_sampling. (time_diff: int = random.choice(time_diff_range))

How did you handle very short videos in sky timelapse?

Thank you!

[universome]

Hi @Jaeyun-Song, thank you! This shouldn't be happening, because we limit the maximum video length by the dataset statistics here. Could you please tell what repo version do you use (i.e. git commit hash) and send the full stack trace for me to see where exactly the error occurs (there are 2 places in the code where sample_frames is called).

Also, just in case, here is our version of the Sky Timelapse dataset: https://disk.yandex.ru/d/7JU3c5mdWQfrHw I've just launched the training from the current repo version on this dataset and it goes without issues (the training has reached 482 kimgs already).

[Jaeyun-Song]

Thank you for your reply! I resolved my problem!

I could train models successfully after pulling repo and using Sky Timelapse dataset from your link.

I think that I might miss some details, could I ask some questions about training details? To reproduce the results in paper, how much images are required to train model on each dataset? (FaceForensics, Sky Timelapse, UCF101) When I training models for 25k imgs, it seems that model does not converge yet.

[universome]

Hi! What do you mean by "does not converge yet"? We train for 25k kimgs everywhere to be comparable with other methods. If you will train for longer — you should be able to obtain better results, though we didn't really try

[Jaeyun-Song]

Thank you for your reply! All of my issues are resolved.

My intention of "does not converge yet" is that the performance is getting better when I train models longer, so I asked the number of images used for the training process.

[universome]

Great, feel free to ask any further questions!