Rewriting `DataSampler` to pytorch `DataLoader` object

sdv-dev / CTGAN

Conditional GAN for generating synthetic tabular data.

Other

1.21k stars 277 forks source link

Rewriting `DataSampler` to pytorch `DataLoader` object #105

Open Baukebrenninkmeijer opened 3 years ago

Baukebrenninkmeijer commented 3 years ago

If the datasampler can be rewritten to a normal pytorch dataloader, we can more easily integrate it with other deep learning frameworks like PyTorch Lightning and Horovod. Both facilitate multi-gpu training and distributed training, which can be super advantageous for larger (/corporate) domains.

fealho commented 3 years ago

This is a good idea but I believe it may take a significant amount of effort to actually implement it. I will shelve it for now, since there are other higher priority tasks, but I agree we should take a look at adding this feature in the future.

Baukebrenninkmeijer commented 3 years ago

@fealho can you label it as such? Or shall I close in the meantime?