How to train it for off-the-shelf BigBiGAN model?

yangyu12 commented 4 years ago

Hi, nice work! I am studying your subsequent work Big GANs Are Watching You: Towards Unsupervised Object Segmentation with Off-the-Shelf Generative Models which is based on the technique proposed in this work. I was attempting to get some interpretable directions for BigBiGAN as described in the paper. What I did is just loading the pretrained BigBiGAN weights provided in this repo: https://github.com/anvoynov/BigGANsAreWatching and then training the model in the same way as training for BigGAN. I suppose that BigGAN and BigBiGAN have the same generator structure so it could have worked. However, it didn't work. The reconstructor was trained very poorly. I wonder what is the correct way to do that. Thanks very much!

training command

python run_train.py \
    --gan_type BigGAN \
    --gan_weights /path/to/BigBiGAN/weights \
    --deformator ortho \
    --out output/BigBiGAN_ortho

anvoynov commented 4 years ago

Hi, @yangyu12 It appears to be more stable to train it with random_init set to False (i.e. start with the coordinate directions), could you try this option? What is the final precision of the discriminator in your run?

yangyu12 commented 4 years ago

Hi @anvoynov. Thanks! I will try your suggestions once my GPU is available.

I didn't finish that run. the last log is 46% [step 46650] | percent: 0.10 | loss: 4.48 | direction_loss: 3.90 | shift_loss: 0.58

I think it's very bad compared to the training log of BigGAN using your provided command: 49% [step 49220] | percent: 0.77 | loss: 1.01 | direction_loss: 0.73 | shift_loss: 0.28

So I think there must be something wrong.

anvoynov commented 4 years ago

This is actually not so bad as BigBiGAN appears to be much less consistent and stable compared to BigGAN. For instance here is my training dynamics for BigBiGAN:

though I still recommend to disable the random_init here.

yangyu12 commented 4 years ago

@anvoynov Apparently I made a mistake. I lacked the UnconditionalBigGAN wrapper. So I copied the codes from your BigGANsAreWatching repo. However, it didn't fix the issue. My training still looks problematic. Should a higher learning rate be used? Could you please help me to see what is wrong?

Training dynamics

args.json

{"shift_scale": null, "min_shift": null, "shift_distribution": null, "deformator_lr": null, "shift_predictor_lr": null, "stnet_lr": null, "n_steps": null, "batch_size": null, "directions_count": null, "max_latent_dim": null, "label_weight": null, "shift_weight": null, "warp_weight": null, "steps_per_log": 200, "steps_per_save": null, "steps_per_img_log": null, "steps_per_backup": null, "truncation": null, "out": "output/BigBiGAN_ortho", "gan_type": "BigBiGAN", "gan_weights": "../BigGANsAreWatching/BigGAN/weights/BigBiGAN_x1.pth", "target_class": [239], "deformator": "ortho", "deformator_random_init": true, "stn": "", "shift_predictor_size": null, "shift_predictor": "ResNet", "shift_distribution_key": null, "seed": 2, "device": 0, "multi_gpu": true, "w_shift": true}

command.sh

run_train.py --gan_type=BigBiGAN --gan_weights=../BigGANsAreWatching/BigGAN/weights/BigBiGAN_x1.pth --deformator=ortho --out=output/BigBiGAN_ortho --multi_gpu=True --deformator_random_init=False --steps_per_log=200

Adapted codes and results

source code: https://github.com/yangyu12/GANLatentDiscovery changes: https://github.com/anvoynov/GANLatentDiscovery/compare/master...yangyu12:master results (only last models are kept): https://drive.google.com/file/d/1r1ZDWDW8aoLcvyzl4uttZ9rbNEd_KAy2/view?usp=sharing

anvoynov commented 4 years ago

Generally, the easiest way to increase the reconstructor precision is to increase the shift scale, e.g. set it equal 8. Have you considered just pass the BigBiGAN's weights with gan_type equal BigGAN? Though it would pass the target_classes tensor to the generator G, it would have no effect as it is a constant class-independent input for BigBiGAN's weights. So you can try to run run_train.py --gan_type=BigGAN --gan_weights=../BigGANsAreWatching/BigGAN/weights/BigBiGAN_x1.pth --deformator=ortho --out=output/BigBiGAN_ortho --multi_gpu=True --deformator_random_init=False --steps_per_log=200 --shift_scale 8 There are some new arguments in your logs like stn and stnet_lr that I can't find in your commits, is there a chance that they impact the training?

yangyu12 commented 4 years ago

Thanks for your recommended command. I will try it soon.

Very sorry for the misleading arguments. I was trying to find some geometric transformation directions in a self-supervised learning manner. If nothing is passed to these arguments, the code would be the same as my commits. To eliminate the confusion, I just deleted that part of codes before I made a commit.

anvoynov / GANLatentDiscovery