About where the mean(ensemble) is calculated

[slyviacassell]

Hi, I find that the function validate() in the segmentation experiment may be wrong. It looks like this. The annotation says

    """Validate segmenter
    Args:
      segmenter (nn.Module) : segmentation network
      val_loader (DataLoader) : training data iterator
      epoch (int) : current epoch
      num_classes (int) : number of classes to consider
    Returns:
      Mean IoU (float)
    """

, but I do not find any operation to calculate the mean value of IoU between the output of RGB and that of depth. It seems just return the IoU of depth, not the mean value. Would you mind giving more details of this?

[yikaiw]

@slyviacassell Hi, as you say "the mean value of IoU between the output of RGB and that of depth", does the "mean value" indicates the ensemble of RGB and Depth? If it does, in Line 346 of main.py, for idx, input_type in #enumerate(input_types + ['ens']):, there is an ensemble term appended to input_types. Therefore the returned IoU should be evaluated on ensemble, not depth.

Besides, note that the network returns a list with three tensors, which are the outputs of the RGB, Depth and Ensemble respectively. The calculation of the ensemble is performed at Line 313-316 in models/model.py. You can have a look at my training logs, shared in the Google Drive. In the training logs, each validation step prints three lines of results, including results on RGB, Depth and Ensemble respectively as well.

Questions are welcome.

[slyviacassell]

Whoops, my fault. I just misunderstand it. Thank you for your prompt reply!

[slyviacassell]

@yikaiw Hi, I found that the ensembled output is detached from the computing graph in the segmentation experiment, which means the ensemble output would not be used to backforward. Would you mind providing some details?

[yikaiw]

Yes, as network parameters have already been updated through RGB and depth branches, the ensemble output is only for updating alpha (ensemble weights), which does not affect the network learning.

[slyviacassell]

Ok, thank you very much!