confusion about nr_cate and maskout in S3.3D_Rotation

[matteorr]

I'm confused about what the variable nr_cate represents for S3.3D_Rotation. The conf.yml file does not have an explanation but only specifies that the default is 12.

Initially I thought it was the number of sign categories that should be predicted by the model, but by looking at the reg_Sexp model it seems like that value is instead len(self.signs)=8.

Also, nr_cate multiplies reg_n_D in the self.head_seq, which is used both for the prediction of the absolute value and the sign of the quaternions, so I don't think it's associated to the sign (at least exclusively). Could you please help me understand?

I'm also confused by the Maskout module. Is it related to nr_cate? If so, how? If not, what does the module do?

Thank you so much!

[leoshine]

Thanks for your question.

nr_cate is short for the “Number of categories”.

• For S1.Viewpoint, we use Pascal3D+ dataset. There’re 12 categories ['aeroplane', 'bicycle', 'boat', 'bottle', 'bus', 'car', 'chair', 'diningtable', 'motorbike', 'sofa', 'train', ‘tvmonitor']. And the model is trained in a class-specific fashion (please refer to the Fig. 1 in our supplemental material), just like anybody else did previously.

• For S3.3D_Rotation, we use ModelNet10-SO3 dataset, where there’re 10 categories ['bathtub', 'bed', 'chair', 'desk', 'dresser', 'monitor', 'night_stand', 'sofa', 'table', 'toilet’]. This is defined in S3.3D_Rotation/lib/datasets/Dataset_Base.py

(For some reason I copy the code from the S1.Viewpoint, and didn’t change the comment “12” to “10", which cause the confusion.) However, since in trainval_workdir.py, we initialize the _net_module (i.e. class reg_Direct_Net or reg_Sflat_Net or reg_Sexp_Net) with argument nr_cate=len(cates), the code is still running with self.nr_cate=10.

-------------------------------------------------[S3.3D_Rotation/trainval_workdir.py] if opt.cates is None: opt.cates = dataset_train.cates # which is the 10 categories from ModelNet10 defined in S3.3D_Rotation/lib/datasets/Dataset_Base.py cates = opt.cates

…

print ('[makenet] nr_cate: ', len(cates)) model = _net_module(nr_cate=len(cates), **_net_kwargs)

Maskout module is not our invention. It’s from “viewpoint and keypoint” paper, where they implemented in c++ as a caffe layer. I simply implemented with pytorch. The purpose of this layer, which again is related to the categories specific prediction, is to mask out the specific prediction for one category from prediction head.

I hope the above helps you clear out the confusion.

[matteorr]

Yeah this clears all the confusion. I just didn't realize (cause I never used this data) that the head is predicting a rotation for all possible classes. Thanks a lot for your answer!