Complete and detailed training evaluation and inference process

[smanman]

Disturbed the author! I would like to ask if I have done run_training.py training the dataset and did not run the run_evaluation_XXX.sh file directly predict_simple.py generating infersTs file and then proceeding to inference_xxx.py to get the final result, is this process correct? The Synapse and Lung datasets do not test whether the set is run_training.py directly inference_xxx.py with the files in the generated validation_raw_postprocessed folder later. Is the final accuracy comparison result the evaluation result or the inference result?

[Amshaker]

Yes, this process is correct because it is based on the testing set.

I verified both ways and the Dice score is the same in both ways.

[Amshaker]

Also, at the end of the training, you can find summary.json in validation_raw_postprocessed folder. This summary contains the final testing accuracy for each organ and the mean of them.

[smanman]

@Amshaker Whether the final test results are generated by inference_xxx.py file or taken directly from the summary.json file in the folder validation_raw_postprocessed the final test accuracy and its average value for each organ Which is the two?

[Amshaker]

Yes, it is the "mean" of all organs, the number of organs is dependent on each dataset.