search

MorenoLaQuatra / audiocaps-download

This package aims at simplifying the download of the AudioCaps dataset.
30 stars 4 forks source link

fatal python error: init_sys_streams: can't initialize sys standard streams #4

Closed louislau1129 closed 4 months ago

louislau1129 commented 4 months ago

When I use n_jobs> 1, it shows: using backend locybackend with n_job concurrent workers. fatal python error: init_sys_streams: can't initilalize sys standard streams, [OS Error] bad file descriptor.

When n_jobs=1, everthing is ok. But I would like to download faster by parallel. Having tried increase the ulimit for allowing the increased open file descriptors. and reduce the verbose level, nothing works. Any idea to solve this parallel issue? Thanks in advance!

MorenoLaQuatra commented 4 months ago

Hi, I think it may be an OS-specific problem, on which OS are you running this? Also you can try to change the backend if needed: here you can change the backend adding a parameter backend='multiprocessing' for example reference.

Of course if it works, it should be done also for validation and test sets.

Hope this helps.