PackNet Slim scale-aware

[tzanis-anevlavis]

Hello! First of all, really inspiring work, thank you for sharing!

I was wondering if you could/have plans to make available a pre-trained version of the scale-aware PackNet Slim?

Moreover, it would be interesting to see, if you have such data, a comparison in terms of speed-up and accuracy between the regular and slim versions! Cheers!!

[VitorGuizilini-TRI]

Our PackNet-SAN model uses PackNet Slim, you can try using only the prediction part (without the SAN module) as a comparison with PackNet.

[tzanis-anevlavis]

Thank you for the reply Vitor. I have two follow-up questions: 1.Scratch the first one. It is resolved. is it possible to extract the weights of PackNet Slim from the PackNet-SAN model? I tried doing so with the following code:

# Load pre-trained ckpt model:
from packnet_sfm.models.model_wrapper import ModelWrapper
from packnet_sfm.utils.config import parse_test_file
from packnet_sfm.utils.load import set_debug
from packnet_sfm.utils.horovod import hvd_init

hvd_init()

# Parse arguments
# config, state_dict = parse_test_file("PackNet01_MR_semisup_CStoK.ckpt")
config, state_dict = parse_test_file("PackNetSAN01_HR_sup_K.ckpt")

# Initialize model wrapper from checkpoint arguments
model_wrapper = ModelWrapper(config)
# Load weights
model_wrapper.load_state_dict(state_dict)
# Extract depth net
dnet = model_wrapper.depth_net

which works with the previous models but not with PackNetSAN as I get the following error:

### Preparing Model
Model: SemiSupModel
DepthNet: PackNetSlimEnc01
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-41-a0dd6f19c5e9> in <module>()
     12 
     13 # Initialize model wrapper from checkpoint arguments
---> 14 model_wrapper = ModelWrapper(config)
     15 # Load weights
     16 model_wrapper.load_state_dict(state_dict)

5 frames
/content/drive/My Drive/Colab Notebooks/packnet-sfm/packnet_sfm/utils/load.py in load_class(filename, paths, concat)
    103             # Return method with same name as the file
    104             return getattr(importlib.import_module(full_path), filename)
--> 105     raise ValueError('Unknown class {}'.format(filename))
    106 
    107 
ValueError: Unknown class PackNetSlimEnc01

The reason why I want to extract the weights or ckpt file of PackNetSlim to convert it to a TRT engine for my application.

Turns out the provided checkpoint has just a different name for the depth network. Renaming it to "PackNetSAN01" solves the above.

2. My second question is regarding the following lines of code: https://github.com/TRI-ML/packnet-sfm/blob/6e3161f60e7161115813574557761edaffb1b6d1/packnet_sfm/utils/depth.py#L302-L304 you seem to scale the output of the network according to ground-truth measurement, even for the scale-aware networks. Why is that?

Thank you in advance!

[VitorGuizilini-TRI]

You can set this flag on and off, depending on your configuration file. So if it is a scale-aware model you can turn the use of groundtruth scale off to get the metric values directly from the network.

[jiaqixuac]

Hi! I want to get the evaluation results.

I'm able to get the metric without depth completion. But when I turn the input_depth_type on, I get worse results. Can you help with this? How to set the config for this? Thanks!