sergeytulyakov / mocogan

MoCoGAN: Decomposing Motion and Content for Video Generation
573 stars 113 forks source link

dim_z_category and video sampling fixes #18

Closed vladyushchenko closed 5 years ago

vladyushchenko commented 5 years ago

Hi, I have found some small issues and would like to provide fixes on:

1) src/data.py - VideoDataset Sampling 16-frame sequence, taking every 2nd frame from shapes dataset cannot be done due to bug in data.py:102. With video_len = 32, self.video_length=16, self.every_nth=2 there are enough frames to sample video with stride 2. In current version, frames from 0 to 15 are sampled, resulting into learning half of the full shape motion (from border to center).

2) src/models.py - VideoGenerator - sample_z_video As suggested in #16, launching training with dim_z_category=0 results into crash.

Output:

{'--batches': '100000',
  "please use transforms.Resize instead.")
 '--dim_z_category': '0',
 '--dim_z_content': '50',
 '--dim_z_motion': '10',
 '--every_nth': '2',
 '--image_batch': '32',
 '--image_dataset': '',
 '--image_discriminator': 'PatchImageDiscriminator',
 '--image_size': '64',
 '--n_channels': '3',
 '--noise_sigma': '0.1',
 '--print_every': '100',
 '--use_categories': False,
 '--use_infogan': False,
 '--use_noise': True,
 '--video_batch': '32',
 '--video_discriminator': 'PatchVideoDiscriminator',
 '--video_length': '16',
 '<dataset>': '../data/shapes',
 '<log_folder>': '../logs/shapes'}
Total number of frames 256000
Traceback (most recent call last):
  File "/home/vlad/PycharmProjects/temp/mocogan/src/train.py", line 130, in <module>
    trainer.train(generator, image_discriminator, video_discriminator)
  File "/home/vlad/PycharmProjects/temp/mocogan/src/trainers.py", line 262, in train
    self.image_batch_size, use_categories=False)
  File "/home/vlad/PycharmProjects/temp/mocogan/src/trainers.py", line 165, in train_discriminator
    fake_batch, generated_categories = sample_fake(batch_size)
  File "/home/vlad/PycharmProjects/temp/mocogan/src/trainers.py", line 236, in sample_fake_image_batch
    return generator.sample_images(batch_size)
  File "/home/vlad/PycharmProjects/temp/mocogan/src/models.py", line 282, in sample_images
    z, z_category_labels = self.sample_z_video(num_samples * self.video_length * 2)
  File "/home/vlad/PycharmProjects/temp/mocogan/src/models.py", line 258, in sample_z_video
    z_category, z_category_labels = self.sample_z_categ(num_samples, video_len)
  File "/home/vlad/PycharmProjects/temp/mocogan/src/models.py", line 234, in sample_z_categ
    classes_to_generate = np.random.randint(self.dim_z_category, size=num_samples)
  File "mtrand.pyx", line 993, in mtrand.RandomState.randint
ValueError: low >= high

Process finished with exit code 1 

Fix: not using z_category in sampling in src/models.py:261

Hope it was helpful Regards, Vlad