Big data loading - time consumption

[CarmiShimon]

Hi mravaneli, Thanks for your great work! I really appreciate your code sharing. I have a question regarding batch loader - as it seems, you load .wav by .wav and it uses only 1 processor which consumes a lot of time. Is there any parallel script that loads the data in multi-processors? Do you recommend on doing so?

Thanks in advance, Carmi

[mravanelli]

Hi Carmi, thank you very much for your interest in my research. In the simple code reported in the repository we didn't use a parallel data loader. However, one can use to pytorch dataloader (https://pytorch.org/docs/stable/data.html#torch.utils.data.DataLoader) that reads all the sentences in parallel.

Best,

Mirco

On Wed, 6 Nov 2019 at 08:32, carmi shimon notifications@github.com wrote:

Hi mravaneli, Thanks for your great work! I really appreciate your code sharing. I have a question regarding batch loader - as it seems, you load .wav by .wav and it uses only 1 processor which consumes a lot of time. Is there any parallel script that loads the data in multi-processors? Do you recommend on doing so?

Thanks in advance, Carmi

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/mravanelli/SincNet/issues/71?email_source=notifications&email_token=AEA2ZVXBRNOUV54JE27WWVLQSLBOFA5CNFSM4JJU6FP2YY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4HXHJCDA, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEA2ZVTJTE3K4TZC2QXTX53QSLBOFANCNFSM4JJU6FPQ .

[CarmiShimon]

Thanks a lot!