mlcommons / GaNDLF

A generalizable application framework for segmentation, regression, and classification using PyTorch
https://gandlf.org
Apache License 2.0
158 stars 79 forks source link

[BUG] Classification issue if "include_validation_with_hausdorff" is explicitly set to false #954

Closed Linardos closed 3 days ago

Linardos commented 3 days ago

Describe the bug

Classification code tries to use Segmentation files if "include_validation_with_hausdorff" is explicitly set to false

To Reproduce

It happened to me as part of changes in FeTS Challenge. It would not recognize the classification commands from the config if the user set include_validation_with_hausdorff=False Steps to reproduce the behavior:

  1. in the updated FeTS Challenge codebase set for classification, (https://github.com/Linardos/Challenge), the main script has include_validation_with_hausdorff=False
  2. Run command the main script
  3. I get an error showing the code tries to work with segmentation files:
    Looping over validation data:   0%|                                                                                                             | 0/12 [00:01<?, ?it/s]
    Traceback (most recent call last):
    File "/home/locolinux2/Challenge/Task_1/FeTS_Challenge_RecEng.py", line 1150, in <module>
    scores_dataframe, checkpoint_folder = run_challenge_experiment(
    File "/home/locolinux2/Challenge/Task_1/fets_challenge/experiment.py", line 496, in run_challenge_experiment
    collaborators[col].run_simulation()
    File "/home/locolinux2/Challenge/Task_1/venv/lib/python3.9/site-packages/openfl/component/collaborator/collaborator.py", line 193, in run_simulation
    self.do_task(task, round_number)
    File "/home/locolinux2/Challenge/Task_1/venv/lib/python3.9/site-packages/openfl/component/collaborator/collaborator.py", line 298, in do_task
    global_output_tensor_dict, local_output_tensor_dict = func(
    File "/home/locolinux2/.local/workspace/src/fets_challenge_model.py", line 43, in validate
    epoch_valid_loss, epoch_valid_metric = validate_network(self.model,
    File "/home/locolinux2/Challenge/Task_1/venv/lib/python3.9/site-packages/GANDLF/compute/forward_pass.py", line 285, in validate_network
    result = step(model, image, label, params, train=True)
    File "/home/locolinux2/Challenge/Task_1/venv/lib/python3.9/site-packages/GANDLF/compute/step.py", line 88, in step
    loss, metric_output = get_loss_and_metrics(image, label, output, params)
    File "/home/locolinux2/Challenge/Task_1/venv/lib/python3.9/site-packages/GANDLF/compute/loss_and_metric.py", line 153, in get_loss_and_metrics
    metric_output[metric] = get_metric_output(
    File "/home/locolinux2/Challenge/Task_1/venv/lib/python3.9/site-packages/GANDLF/compute/loss_and_metric.py", line 30, in get_metric_output
    metric_output = metric_function(prediction, target, params).detach().cpu()
    File "/home/locolinux2/Challenge/Task_1/venv/lib/python3.9/site-packages/GANDLF/metrics/segmentation.py", line 61, in multi_class_dice
    current_dice = dice(prediction[:, i, ...], target[:, i, ...])
    IndexError: too many indices for tensor of dimension 1

Crude Solution

It was solved by setting include_validation_with_hausdorff=True, and classification runs normally.

Linardos commented 3 days ago

actually this is an issue for FeTS codebase not GaNDLF itself.