Currently, TorchDataset makes many function calls, this is most likely not efficient. It should either have a lazy loading functionality that allows it to load from csv per batch using the CsvLoader and then make the changes on the fly (if low memory) OR should transform all the data into tensors in setup.
Currently, TorchDataset makes many function calls, this is most likely not efficient. It should either have a lazy loading functionality that allows it to load from csv per batch using the CsvLoader and then make the changes on the fly (if low memory) OR should transform all the data into tensors in setup.