kiyo-masui / bitshuffle

Filter for improving compression of typed binary data.
Other
219 stars 76 forks source link

Decompression slow downs for "too many" threads #138

Open jonwright opened 1 year ago

jonwright commented 1 year ago

It seems the openmp locks and (dynamic,1) overhead can become important for machines with large numbers of cores. For decompression, I could see some improvements using static scheduling:

20230206_bench

Perhaps there is a better way to overcome this problem? Anyway, I will try to send you a pull request for dicussion.