search

BorgwardtLab / proteinshake

Protein structure datasets for machine learning.
https://proteinshake.ai
BSD 3-Clause "New" or "Revised" License
101 stars 9 forks source link

Parallel parsing does not work #66

Closed timkucera closed 1 year ago

timkucera commented 2 years ago

If you compare the single-job vs parallelization times they are roughly the same. I tried chunking to reduce the overhead but still the same. I can also see that the cores are not utilized.

timkucera commented 2 years ago

might be an IO bottleneck that cannot be solved

timkucera commented 1 year ago

was an IO problem on our server, fixed with 2c8433bfb4c8264e7cccc4afe06268e47d580abd