Confusion for Training and Test

[XiHuYan]

You created data generator DG_TRAIN and DG_TEST in both the unet.py and dist.py files:

DG_TRAIN = DataGenMulti(PATH, split='train', crop = 16, size=SIZE,
                       transforms=transform_list, UNet=True, mean_file=None, num=TEST_PATIENT)
TEST_PATIENT = ["test"]
DG_TEST  = DataGenMulti(PATH, split="test", crop = 1, size=(500, 500), 
                       transforms=transform_list_test, UNet=True, mean_file=MEAN_FILE, num=TEST_PATIENT)

but you only used DG_TEST for training.

if SPLIT == "train":
        model.train(DG_TEST)   # ???????

Does that mean you are training with test sets?

[XiHuYan]

your code just drives me crazy.

[PeterJackNaylor]

Hi,

Someone posted the same question in this issue https://github.com/PeterJackNaylor/DRFNS/issues/7

When you create the DataGenMulti object he expects to find the training data in PATH. At training time, you can however feed in an optional DataGenMulti argument to be evaluated at test time.

[XiHuYan]

Does that mean you use the test set in training to select the best hyperparameters? Then you use the selected hyperparameters to test your model with the same test set and compare your results with other methods?

[PeterJackNaylor]

No I do not, I use the validation procedure as written in the paper. The file you should be looking at is the nextflow file. You will see that there are three procedures training, testing and validating (What I call testing is what you call validation I think).

In the procedure training, I train. In the procedure testing, I take the weights in the train procedure and I evaluate with the subset of hyper-parameters. In the procedure validing, I take the model that maximise the testing procedure and I give an unbiais version of the performance.

[XiHuYan]

Well, thank you for your reply. Maybe you can make the code more intuitive later on.