search

shubham-goel / ucmr

Code for the ECCV2020 paper "Shape and Viewpoint without Keypoints".
https://shubham-goel.github.io/ucmr/
148 stars 11 forks source link

Reproducing results for other classes #21

Closed akcheu closed 2 years ago

akcheu commented 3 years ago

Hi @shubham-goel,

I’m currently trying to reproduce the results for the PASCAL3D+ car class as shown in your paper, however I have been getting strange outputs from the model (see images below). The texture prediction seems to be completely inaccurate across the board, especially compared to your results. I got similar results with the airplane and shoe classes as well. Do you have any idea why this might be happening?

Here are the steps I followed:

  1. Downloaded the car template mesh and annotation files from the CSM paper

  2. Symmetrized mesh and rotated along y-z plane in Blender

  3. Recomputed camera multiplex using car template shape with K=8 Cameras initialized at zero elevation. These are the specifications I used in my cam_init flag file: --batch_size=12 --num_multipose_el=1 --dataset=p3d --nopred_shape --nopred_pose --nopred_texture --shape_path=cachedir/template_shape/car.obj --nois_train --num_epochs=1 --optimizeSteps=75 --print_freq=1 --display_freq=5 --nodrop_last_batch --save_camera_pose_dict --save_partial_camera_pose_dict --renderer=nmr

  4. Trained for 21 epochs with car template shape: --batch_size=6 --num_multipose_el=1 --pred_shape --pred_texture --shape_path=cachedir/template_shape/car.obj --dataset=p3d --max_grad_norm=100 --optimizeAlgo=adam --learning_rate=0.0001 --optimizeLR=0.0001 --rend_mask_loss_wt=10 --texture_loss_wt=1 --deform_loss_wt=1 --graphlap_loss_wt=0.5 --edge_loss_wt=0 --num_epochs=21 --cameraPoseDict=cachedir/logs/car_init_campose/stats/campose_0.npz

I have not completed the third training step for 400 epochs after pruning the camera multiplex. Even so, I expected better results after 21 epochs and am not sure if I'm doing something wrong or missing a step. I was wondering if you could provide the exact specifications and hyperparameters that you used for the camera multiplex initialization and training steps for these classes? If possible, could you also provide a pre-trained network for these classes similar to the CUB demo?

Your response is greatly appreciated!

Thanks, Allen

<img src="https://user-images.githubusercontent.com/69490491/117643400-35488a80-b13d-11eb-8678-9fe73753c367.png" alt="car1" width="400"/>

car1

<img src="https://user-images.githubusercontent.com/69490491/117643913-c6b7fc80-b13d-11eb-9d15-817dcbe21da3.png" alt="car2" width="400"/>

car2

<img src="https://user-images.githubusercontent.com/69490491/117644117-00890300-b13e-11eb-947e-6f1e8084b90f.png" alt="car3" width="400"/>

car3
shubham-goel commented 3 years ago

Hi Allen.

Thanks for the detailed issue, I'm not sure why this is happening. Could be that the UV mapping from sphere-unprojecting the vertices gets messed up.

Before we debug why it's not working with your template mesh + config parameters, can you try using my template shape and training recipe (for Pascal Cars) from the latter half of this doc? It also contains a pretrained model for cars.

shubham-goel commented 2 years ago

Closing due to inactivity. Please reopen if this remains unresolved!