``evaluation_prediction.py`` gets me an error.

[hihunjin]

Hi.

I ran

python kits21/annotation/sample_segmentations.py
python kits21/evaluation/evaluate_predictions.py output

Then it gets me

multiprocessing.pool.RemoteTraceback:
"""
Traceback (most recent call last):
  File "/opt/conda/lib/python3.8/multiprocessing/pool.py", line 125, in worker
    result = (True, func(*args, **kwds))
  File "/opt/conda/lib/python3.8/multiprocessing/pool.py", line 48, in mapstar
    return list(map(*args))
  File "/workspace/kits21/kits21/evaluation/metrics.py", line 200, in evaluate_predicted_file_on_samples
    raise RuntimeError('segmentation_samples folder missing. Please run kits21/annotation/sample_segmentations.py')
RuntimeError: segmentation_samples folder missing. Please run kits21/annotation/sample_segmentations.py
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "kits21/evaluation/evaluate_predictions.py", line 29, in <module>
    evaluate_predictions_on_samples(args.folder_with_predictions, args.num_processes)
  File "/workspace/kits21/kits21/evaluation/metrics.py", line 168, in evaluate_predictions_on_samples
    metrics = p.map(evaluate_predicted_file_on_samples, predicted_segmentation_files)
  File "/opt/conda/lib/python3.8/multiprocessing/pool.py", line 364, in map
    return self._map_async(func, iterable, mapstar, chunksize).get()
  File "/opt/conda/lib/python3.8/multiprocessing/pool.py", line 771, in get
    raise self._value
RuntimeError: segmentation_samples folder missing. Please run kits21/annotation/sample_segmentations.py

I read link, and I'm not sure why I should run python kits21/annotation/sample_segmentations.py. Okay you said, this is about 5-fold CV. And the description in the kits21/evaluation/evaluate_predictions.py says whatever in the directory output will give us the result.

[hihunjin]

I solved this by adding --use_maj_voting_as_gt