Training Dataset Preparation Issues and Enhancements

ztsv-av / spellbook

Shortcuts to your ML workflow!

4 stars 1 forks source link

Training Dataset Preparation Issues and Enhancements #12

Closed ztsv-av closed 3 years ago

ztsv-av commented 3 years ago

[x] normalize data again (or do not normalize before that, do not know) after permutations
[x] load data before augmentation... and do not reinitialize it, pass it as a parameter to training function
[ ] prepareClassificationDataset and prepareDetectionDataset should be almost in the same format - decide if to use batches of data or whole dataset

ztsv-av commented 3 years ago

I do not think it is possible to make classification and detection dataset load in the same format. We use distributed dataset in classification to train on two GPUs. However, in detection we only use 1 GPU, meaning that we can load data in batches, unlike in classification.