mlcommons / GaNDLF

A generalizable application framework for segmentation, regression, and classification using PyTorch
https://gandlf.org
Apache License 2.0
160 stars 79 forks source link

[BUG] inpainting metrics are not computed correctly #957

Open neuronflow opened 6 days ago

neuronflow commented 6 days ago

GaNDLF produces metrics that are different from our official inpainting package (https://pypi.org/project/inpainting/).

To compute metrics with the official package:

pip install inpainting
from inpainting.challenge_metrics_2023 import generate_metrics, read_nifti_to_tensor

def compute_image_quality_metrics(
    prediction: str,
    healthy_mask: str,
    reference_t1: str,
    voided_t1: str,
) -> dict:
    print("computing metrics!")
    print("prediction:", prediction)
    print("healthy_mask:", healthy_mask)
    print("reference_t1:", reference_t1)
    print("voided_t1:", voided_t1)

    prediction_data = read_nifti_to_tensor(prediction)
    healthy_mask_data = read_nifti_to_tensor(healthy_mask).bool()
    reference_t1_data = read_nifti_to_tensor(reference_t1)
    voided_t1_data = read_nifti_to_tensor(voided_t1)

    metrics = generate_metrics(
        prediction=prediction_data,
        target=reference_t1_data,
        normalization_tensor=voided_t1_data,
        mask=healthy_mask_data,
    )

    return metrics

if __name__ == "__main__":
    official_metrics = compute_image_quality_metrics(
    prediction="path_to_prediction.nii.gz",
    healthy_mask"path_to_healthy_mask.nii.gz",
    reference_t1"path_to_reference.nii.gz",
    voided_t1"path_to_voided.nii.gz",
    )

    print(official_metrics)

@MarcelRosier will upload some test data to reproduce.

MarcelRosier commented 6 days ago

Test data: INP-BraTS-GLI-00000-000.zip (The Prediction was generated using last years winning algorithm)