To make the transforms easier to work with. We should pre-split the data into training, validation, and testing.
The testing data is a single image and is already split off. The training data needs to be randomly split and this split needs to be saved somewhere.
This is required so that we can use no image transformations during validation and also get accurate class frequencies during training. If we use a single dataset and then do a split we run into the following issues:
Data leakage from computing the class frequencies using validation data
Validation data does not reflect real-world data (it has been randomly augmented)
We should implement TrainHuBMAP, ValidHuBMAP, and TestHuBMAP datasets instead of the single HuBMAP dataset.
To make the transforms easier to work with. We should pre-split the data into training, validation, and testing.
The testing data is a single image and is already split off. The training data needs to be randomly split and this split needs to be saved somewhere.
This is required so that we can use no image transformations during validation and also get accurate class frequencies during training. If we use a single dataset and then do a split we run into the following issues:
We should implement
TrainHuBMAP
,ValidHuBMAP
, andTestHuBMAP
datasets instead of the singleHuBMAP
dataset.