Implementing lazy loading will help Protein GAB run on more hardware. Currently loading the 64x64 training dataset into memory takes up approximately 10GB which is not acceptable for most hardware configurations.
In order to implement lazy loading the following must be accomplished:
estimate RAM needs
cache based on loading the data in chunks
let the user choose how much memory to reserve for Protein GAN and load that much of the dataset.
This will be handled as part of a custom pytorch dataset class.
Implementing lazy loading will help Protein GAB run on more hardware. Currently loading the 64x64 training dataset into memory takes up approximately 10GB which is not acceptable for most hardware configurations.
In order to implement lazy loading the following must be accomplished:
This will be handled as part of a custom pytorch dataset class.