search

crtie / Leveraging-SE-3-Equivariance-for-Learning-3D-Geometric-Shape-Assembly

This is the official implementation of the paper "Leveraging SE(3)-Equivariance for Learning 3D Geometric Shape Assembly" (ICCV 2023)
https://crtie.github.io/SE-3-part-assembly/
41 stars 2 forks source link

[BUG] RuntimeError: mat1 and mat2 shapes cannot be multiplied (93x1024 and 512x1024) #4

Closed panxy16 closed 8 months ago

panxy16 commented 8 months ago

Hi, thank you for sharing the code.

The bug occurs when I edit the config file to use the VN_Regressor to predict the pose. So I wonder if the program supports the VN_Regressor. Here's where the bug occurs: File "Leveraging-SE-3-Equivariance-for-Learning-3D-Geometric-Shape-Assembly/BreakingBad/multi _part_assembly/models/vnn/network.py", line 217, in forward R_6d, trans = self._predict_pose( File "Leveraging-SE-3-Equivariance-for-Learning-3D-Geometric-Shape-Assembly/BreakingBad/multi _part_assembly/models/vnn/network.py", line 167, in _predict_pose valid_R_6d, valid_trans = self.pose_predictor( ... File "Leveraging-SE-3-Equivariance-for-Learning-3D-Geometric-Shape-Assembly/BreakingBad/multi _part_assembly/models/vnn/modules.py", line 152, in forward f = self.fc_layers(x) ... File "Leveraging-SE-3-Equivariance-for-Learning-3D-Geometric-Shape-Assembly/BreakingBad/multi _part_assembly/models/vnn/vn_layers.py", line 33, in forward x_out = self.map_to_feat(x.transpose(1,-1)).transpose(1,-1) File "/root/miniconda3/envs/assemblySE3/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1102, in _ca ll_impl return forward_call(*input, **kwargs) File "/root/miniconda3/envs/assemblySE3/lib/python3.8/site-packages/torch/nn/modules/linear.py", line 103, in forw ard return F.linear(input, self.weight, self.bias) File "/root/miniconda3/envs/assemblySE3/lib/python3.8/site-packages/torch/nn/functional.py", line 1848, in linear return torch._C._nn.linear(input, weight, bias) RuntimeError: mat1 and mat2 shapes cannot be multiplied (93x1024 and 512x1024)

crtie commented 8 months ago

Thank you for your interest in our work!

In our method, the rotation regressor is not equivariant (as shown in the top-right "Pose Regressor" part of Figure 2). From our empirical experiments, an equivariant rotation regressor limits the degrees of freedom in pose estimation, reducing prediction accuracy. The VN_Regressor in our code was only used to test performance during our development process and is not a part of our final algorithm framework; thus, we didn't set it in the released configurations.

panxy16 commented 8 months ago

Thanks for your reply!