trying to handle num_workers and dataset slicing in IterableDataModule

dsgt-birdclef / birdclef-2022

Code for the BirdCLEF 2022 competition by the DS@GT team

2 stars 3 forks source link

trying to handle num_workers and dataset slicing in IterableDataModule #27

Closed bran22 closed 2 years ago

bran22 commented 2 years ago

I gotta say... I'm not really sure I understand what's going on in get_motif_pairs (Is the implementation done? Seems like it might be missing distant samples?).

However, I did try and contribute some to the IterableDataset and IterableDataModule stuff. Hopefully this is on the right track?

bran22 commented 2 years ago

Nope, feel free to go ahead and merge and take back over on the branch. Sounds like you have a decent vision for what you want to do and I don't want to be a bottleneck!