search

autonomousvision / convolutional_occupancy_networks

[ECCV'20] Convolutional Occupancy Networks
https://pengsongyou.github.io/conv_onet
MIT License
830 stars 113 forks source link

Few quick questions about training model #25

Closed EternityDS closed 3 years ago

EternityDS commented 3 years ago

Hi Songyou,

Thanks for sharing your brilliant work! I would like to ask two quick questions here. Hopefully this will not take you too much time :)

  1. I'm running the 3-plane & grid models with default setting on synthetic scene dataset. I noticed that 3-plane model can converge after 350k iterations but grid model keeps growing up even after 500k iterations. Training 500k iterations actually takes a long time(5-6 days on one Tesla V100 gpu). I also noticed the default learning rate is 1e-4 and it never changes. I'm wondering whether it is possible to have a better learning rate schedule for scene dataset? I attached my iou curve here for your information(gray for grid and red for 3-plane):

    image

  2. I noticed that you set y-axis as the up-axis for all the data. May I ask does it matter if I use z-axis as the up-axis?

pengsongyou commented 3 years ago

Hi @EternityDS,

Thanks for your words and sorry that I forgot to reply.

  1. Good point! It is indeed very slow with the grid model because of the 3D CNN and group normalization used in 3D U-Net. For the learning rate, I was following the occupancy network and fixed 1e-4 the whole time. You can definitely try to have a better learning rate scheduling, but I personally never tried it.
  2. I am doing so because, in our processed ShapeNet dataset, y-axis corresponds to the up-axis :)

Best, Songyou

EternityDS commented 3 years ago

Thanks for your reply! This helps me a lot!

ngjuping commented 2 years ago

Hello, @EternityDS , what do you mean the up axis is y-axis? Would u mind elaborate further?