734 large dice

[mathpluscode]

Description

The main issue that dice score > 1 because the added metrics were not normalized by the global batch size.

In this PR, we also double-checked/refactored the implementation of several loss functions and added extensive tests to verify.

Fixes #734

Type of change

What types of changes does your code introduce to DeepReg?

Please check the boxes that apply after submitting the pull request.

• [x] Bugfix (non-breaking change which fixes an issue)
• [ ] Code style update (formatting, renaming)
• [x] Refactoring (no functional changes, no api changes)
• [ ] Documentation Update (fix or improvement on the documentation)
• [ ] New feature (non-breaking change which adds functionality)
• [ ] Other (if none of the other choices apply)

Checklist

Please check the boxes that apply after submitting the pull request.

If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your code.

• [x] I have installed pre-commit using pre-commit install and formatted all changed files. If you are not certain, run pre-commit run --all-files.
• [x] My commits' message styles matches our requested structure, e.g. Issue #<issue number>: detailed message.
• [x] I have updated the change log file regarding my changes.
• [x] I have added tests that prove my fix is effective or that my feature works
• [x] I have added necessary documentation (if appropriate)

[mathpluscode]

@YipengHu feel free to try again?

[codecov[bot]]

Codecov Report

Merging #736 (2c573f9) into main (bbe77ba) will not change coverage. The diff coverage is 100.00%.

@@            Coverage Diff            @@
##              main      #736   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files           39        39           
  Lines         2455      2464    +9     
=========================================
+ Hits          2455      2464    +9     

Impacted Files	Coverage Δ
deepreg/dataset/loader/interface.py	100.00% <ø> (ø)
deepreg/dataset/preprocess.py	100.00% <ø> (ø)
deepreg/loss/deform.py	100.00% <100.00%> (ø)
deepreg/loss/image.py	100.00% <100.00%> (ø)
deepreg/loss/label.py	100.00% <100.00%> (ø)
deepreg/model/network.py	100.00% <100.00%> (ø)
deepreg/predict.py	100.00% <100.00%> (ø)
deepreg/train.py	100.00% <100.00%> (ø)
deepreg/util.py	100.00% <100.00%> (ø)

---

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update bbe77ba...2c573f9. Read the comment docs.

[mathpluscode]

@YipengHu I checked the implementation of loss and metric, i.e.

• MSE loss https://github.com/tensorflow/tensorflow/blob/v2.4.1/tensorflow/python/keras/losses.py#L267-L322
• MSE metric https://github.com/tensorflow/tensorflow/blob/v2.4.1/tensorflow/python/keras/metrics.py#L2378-L2410

They are different classes calling the same functions.

Then regarding the value under multiple GPUs, the doc is here https://www.tensorflow.org/tutorials/distribute/custom_training#define_the_loss_function

So, in short, there are a lot of work to be done to make it right lol

[mathpluscode]

For now, I removed binary dice from metrics, as we were using loss class as metric.

Will transform existing loss functions and add metric back.