FID 188 on shapenet motorbike using the pretrained model

[hitsz-zuoqi]

1、I download the pretrained model of class motorbike of shapenet, but when i test using the following command, i can only get fid:188, command below:

python train_3d.py --outdir=save_inference_results/shapenet_motorbike --gpus=8 --batch=32 --gamma=40 --data_camera_mode shapenet_motorbike --dmtet_scale 1.0 \ --use_shapenet_split 1 --one_3d_generator 1 --fp32 0 --inference_vis 1 --resume_pretrain ./pretrained_model/shapenet_motorbike.pt --inference_compute_fid 1 \ --data=./render_shapenet/img/03790512 \ --camera_path ./render_shapenet/camera/

metrics below: {"results": {"fid50k": 188.33846737901956}, "metric": "fid50k", "total_time": 315.1733651161194, "total_time_str": "5m 15s", "num_gpus": 8, "snapshot_pkl": "../../../pretrained_model/shapenet_motorbike.pt", "timestamp": 1672802728.9199426} {"results": {"fid50k": 188.33846737901956}, "metric": "fid50k", "total_time": 315.4739305973053, "total_time_str": "5m 15s", "num_gpus": 8, "snapshot_pkl": "../../../pretrained_model/shapenet_motorbike.pt", "timestamp": 1672802728.9199436} {"results": {"fid50k": 188.33846737901956}, "metric": "fid50k", "total_time": 315.8814013004303, "total_time_str": "5m 16s", "num_gpus": 8, "snapshot_pkl": "../../../pretrained_model/shapenet_motorbike.pt", "timestamp": 1672802728.9199443} {"results": {"fid50k": 188.33846737901956}, "metric": "fid50k", "total_time": 315.00143790245056, "total_time_str": "5m 15s", "num_gpus": 8, "snapshot_pkl": "../../../pretrained_model/shapenet_motorbike.pt", "timestamp": 1672802728.919946} {"results": {"fid50k": 188.33846737901956}, "metric": "fid50k", "total_time": 315.74815583229065, "total_time_str": "5m 16s", "num_gpus": 8, "snapshot_pkl": "../../../pretrained_model/shapenet_motorbike.pt", "timestamp": 1672802728.9199429} {"results": {"fid50k": 188.33846737901956}, "metric": "fid50k", "total_time": 313.818487405777, "total_time_str": "5m 14s", "num_gpus": 8, "snapshot_pkl": "../../../pretrained_model/shapenet_motorbike.pt", "timestamp": 1672802728.9199443} {"results": {"fid50k": 188.33846737901956}, "metric": "fid50k", "total_time": 321.5122573375702, "total_time_str": "5m 22s", "num_gpus": 8, "snapshot_pkl": "../../../pretrained_model/shapenet_motorbike.pt", "timestamp": 1672802728.920031} {"results": {"fid50k": 188.33846737901956}, "metric": "fid50k", "total_time": 315.77975964546204, "total_time_str": "5m 16s", "num_gpus": 8, "snapshot_pkl": "../../../pretrained_model/shapenet_motorbike.pt", "timestamp": 1672802728.920079}

2、another phenomenon I observe in the training process is that it often collaspe after reaching the best metrics，do you know why？

[SteveJunGao]

Hi @hitsz-zuoqi ,

Thanks for trying our pretrained model!

Re 1: can you show some images in your test set (rendered from blender script) and the images generated from the pretrained model, such that we can double check? It seems there could be a gap between these two set images.

Re 2: yes, I also observe this phenomenon in the training process, this is likely due to discriminator on the rendered 2D mask is a bit unstable (since it only looks at a binary image)

[hitsz-zuoqi]

Hi @hitsz-zuoqi ,

Thanks for trying our pretrained model!

Re 1: can you show some images in your test set (rendered from blender script) and the images generated from the pretrained model, such that we can double check? It seems there could be a gap between these two set images.

Re 2: yes, I also observe this phenomenon in the training process, this is likely due to discriminator on the rendered 2D mask is a bit unstable (since it only looks at a binary image)

thanks, i have generated the pictures using only one GPU and the metric is fine(50.6). I have checked the pictures generated by multi-gpus and it looks fine, I think the statistics may be evaluated for each single piece of GPU so the metric will be destroyed if we evaluate under multi-gpu setting.

[SteveJunGao]

Ah, yes, the inference code to compute FID is only tested with a single GPU (I might make some assumptions when writing the inference code).

On the other side, if you running the training script, it will compute FID on multiGPU, and this should work