zhanglab-aim / cancer-net

Diagnosing cancers using deep learning.
GNU General Public License v2.0
2 stars 0 forks source link

Add sampler for deterministic sampling of test set #26

Closed Chris-Pedersen closed 1 year ago

Chris-Pedersen commented 1 year ago

I want to be able to train models with random initial weights and train/valid/test splits, but test predictions on a deterministic ordering of the test set for a given train/valid/test seed, as we are interested in comparing the predictions between models at each specific data sample. Amazingly I couldn't find a torch sampler able to do this, without fixing the global random seed (the SequentialSampler, which I thought would do this, simply returns indices based on the length of the list of indices, it doesn't return a non-shuffled list of the indices themselves, which seems totally useless). So have added one called SequenceSampler.

ttesileanu commented 1 year ago

Turns out we can use the generator option on SubsetRandomSampler, so this is not needed.