VSS loss not found in dictionary when attempting training

[damiankucharski]

Hello @YixingHuang, I am now trying to train your model on my data to compare how the performance will change compared to just using your pretrained model. I followed the steps from readme but I am getting the following error.

CONFIG: The configuration file for the [session] was loaded from:~/DeepMedicPlus/DeepMedicPlus/examples/configFiles/deepMedicPlus/train/trainConfig.cfg

ERROR: Caught exception from main process: 'vss'
Traceback (most recent call last):
  File "~/DeepMedicPlus/DeepMedicPlus/deepMedicRun", line 175, in <module>
    _ = session.compile_session_params_from_cfg(model_params)
  File "~/DeepMedicPlus/DeepMedicPlus/deepmedic/frontEnd/trainSession.py", line 78, in compile_session_params_from_cfg
    self._params = TrainSessionParameters(self._log,
  File "~/DeepMedicPlus/DeepMedicPlus/deepmedic/frontEnd/configParsing/trainSessionParams.py", line 518, in __init__
    assert True in [self.losses_and_weights[k] is not None for k in ["xentr", "iou", "dsc", "vss"]]
  File "~/DeepMedicPlus/DeepMedicPlus/deepmedic/frontEnd/configParsing/trainSessionParams.py", line 518, in <listcomp>
    assert True in [self.losses_and_weights[k] is not None for k in ["xentr", "iou", "dsc", "vss"]]
KeyError: 'vss'

Finished

What may be causing the issue? It looks as if there is a bug in the code but I assume I may be doing something wrong since you used your code to train your own models...

[YixingHuang]

Hi DK, @damiankucharski I cleaned the codes after my experiments. There might be bugs. I just updated one line here Line 119: cost += self._losses_and_weights["vss"] * cfs.vss(p_y_given_x, y_gt)

In addition, in your train config, please make sure that losses_and_weights = {"xentr": 0.5, "iou": None, "dsc": None, "vss":0.5} where vss is chosen.

Please try whether it works now.

[damiankucharski]

Hi @YixingHuang,

Sorry for late reply. I've run some experiments and played a little with the alpha parameter in the loss function. I also compared the results with nnDetection and nnUnet. It seems that even the models set up with very high precision in mind (I tried alpha values as low as 0.05) have lower precision that nnUNet models for example, however they do tend to find some lesions that nnUNet and nnDetection have trouble finding. I am going to perform more experiments and investigate further but it seems that these architectures have tendencies to work a little better for different kinds of lesions. Thank you very much for your help.

Regards, Damian Kucharski

[YixingHuang]

Hi DK @damiankucharski, thank you for sharing your update. It is a complex nonconvex optimization. Different architectures, different optimizers will always find slightly different local minima. Good to hear that our model can find some difficult brain metastases.

[YixingHuang]

Since the original bug has been fixed, I simply close this issue. If you have more issues or want to share your performance evaluation, feel free to create a new issue.