search

GANWANSHUI / GaussianOcc

GaussianOcc: Fully Self-supervised and Efficient 3D Occupancy Estimation with Gaussian Splatting
https://ganwanshui.github.io/GaussianOcc/
Apache License 2.0
211 stars 10 forks source link

Clarification on Stages of training #30

Open seamie6 opened 3 weeks ago

seamie6 commented 3 weeks ago

You have broken down training into 2 steps: Stage 1, and Stage 2 In some responses you say Stage 1 code has not been released yet. But if this is the case then how is it possible to train the model. Is it that a checkpoint for Stage 1 is embedded in the code? If it is now included, where exactly is Stage 1 training taking place, and then Stage 2 also? As to my knowledge they are done separately. Thank you.

GANWANSHUI commented 3 weeks ago

Yes, stages 1 and 2 are separate. For now, we have released the 6d pose network (trained from stage 1) for stage 2.

seamie6 commented 2 weeks ago

Could you clarify why we have this function inside runner.py

predict_poses(self, inputs, features)

which seems to be trained alongside Stage 2? I thought this was part of Stage 1 no? The models self.models["pose_encoder"] and self.models["pose"] inside of it seem to have trainable params so this is not just evaluation.

seamie6 commented 2 weeks ago

An additional question I have is in occupancy_decoder.py, in the get_voxel(self, features, inputs) function, which coordinate frame is Voxel_feat stored in reference to? Thanks

GANWANSHUI commented 2 weeks ago

Could you clarify why we have this function inside runner.py

predict_poses(self, inputs, features)

which seems to be trained alongside Stage 2? I thought this was part of Stage 1 no? The models self.models["pose_encoder"] and self.models["pose"] inside of it seem to have trainable params so this is not just evaluation.

It is not trainable and only for providing the 6d pose from stage 1.

GANWANSHUI commented 2 weeks ago

An additional question I have is in occupancy_decoder.py, in the get_voxel(self, features, inputs) function, which coordinate frame is Voxel_feat stored in reference to? Thanks

it was referred to as the ego center.

seamie6 commented 2 weeks ago

In predict_poses() printing out:

self.models["pose_encoder"].training

return True. Should it not be False if we do not want to train it?