Error while training Packnet-Sfm on EuRoC

[yhabib29]

Hi,

I could not train Packnet-Sfm network on this repo, I get an error when the network tries to infer pose:

File "/workspace/vidar/vidar/arch/models/depth/SelfSupervisedModel.py", line 76, in forward
    pose = self.compute_pose(rgb, self.networks['pose'], tgt=0, invert=True)
  File "/workspace/vidar/vidar/arch/models/BaseModel.py", line 39, in compute_pose
    for idx in ctx}
  File "/workspace/vidar/vidar/arch/models/BaseModel.py", line 39, in <dictcomp>
    for idx in ctx}
  File "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py", line 1102, in _call_impl
    return forward_call(*input, **kwargs)
TypeError: forward() got an unexpected keyword argument 'invert'

I used the following configuration:

wrapper:
    recipe: wrapper|default
    max_epochs: 1
arch:
    model:
        file: depth/SelfSupervisedModel
    networks:
        depth:
            recipe: networks/packnet|default
            depth_range: [0.1,20.0]
        pose:
            recipe: networks/conv_pose_net|default
    losses:
        reprojection:
            recipe: losses/reprojection|default
        smoothness:
            recipe: losses/smoothness|default
evaluation:
    depth:
        recipe: evaluation/depth|euroc_resize
optimizers:
    depth:
        recipe: optimizers|adam_20_05
    pose:
        recipe: optimizers|adam_20_05
datasets:
    train:
        recipe: datasets/euroc|train_selfsup
    validation:
        recipe: datasets/euroc|validation

Indeed, the _computepose function in BaseModel.py is not meant for Packnet module. In order to make it work, I had to change the following lines in SelfSupervisedModel.py based on the previous repo of packnet-sfm:

index 68c360b..2656cb1 100644
--- a/vidar/arch/models/depth/SelfSupervisedModel.py
+++ b/vidar/arch/models/depth/SelfSupervisedModel.py
@@ -7,6 +7,7 @@ from vidar.arch.models.BaseModel import BaseModel
 from vidar.arch.models.utils import make_rgb_scales, create_cameras
 from vidar.utils.data import get_from_dict
 from vidar.utils.config import cfg_has
+from vidar.geometry.pose import Pose

 class SelfSupervisedModel(BaseModel, ABC):
@@ -72,7 +73,11 @@ class SelfSupervisedModel(BaseModel, ABC):
             assert 'pose' in batch, 'You need input pose'
             pose = batch['pose']
         elif 'pose' in self.networks:
-            pose = self.compute_pose(rgb, self.networks['pose'], tgt=0, invert=True)
+            # pose = self.compute_pose(rgb, self.networks['pose'], tgt=0, invert=True)
+            # pose = self.networks['pose'](rgb[0], [rgb[key] for key in rgb.keys() if key != 0])
+            pose_vec = self.networks['pose'](rgb[0], [rgb[-1], rgb[1]])
+            pose = {-1: Pose.from_vec(pose_vec[0, 0].unsqueeze(0), 'euler'),
+                     1: Pose.from_vec(pose_vec[0, 1].unsqueeze(0), 'euler')}
             predictions['pose'] = pose
         else:
             pose = None

The inference worked so I tried to train the network on EuRoC/V1_01 easy scene. I wanted to overfit the network just to check if the pipeline works. However, the results are really bad. How am I suppose to train Packnet-Sfm network on EuRoC using this repo ?

Thanks for your help !