inaos / iron-array

2 stars 0 forks source link

Avoid duplicating work when evaluating uncompressible result #542

Open FrancescAlted opened 2 years ago

FrancescAlted commented 2 years ago

When Blosc tries to compress a chunk, and the out buffer does not fit in the original size, a MEMCPY flag is activated, and compression restarts. And yes, that means that the prefilter (and hence, the computation) is executed twice.

This is not terribly bad, as the user can always disable compression manually, avoiding the overhead in these cases. However, it would be nice if we can make BTune to disable compression automatically in these cases.