search

tammoippen / plotille

Plot in the terminal using braille dots.
MIT License
419 stars 19 forks source link

Index out of bounds for hist() on negative values #35

Closed audatic-henning closed 4 years ago

audatic-henning commented 4 years ago

Negative values in histogram sometimes seem to lead to negative bin indices that can underflow (and shouldnt be there in the first place).

I believe the fix is in plotille/_util in hist() around line 85 to change the computation of x_idx to

        x_idx = max(min(bins - 1, int(delta // xwidth)), 0)

(the max is new). Alas I don't have time to thoroughly test and/or create a PR right now, maybe someone else can pick it up

audatic-henning commented 4 years ago

I should add I'm using numpy arrays of int16s, seems even with the fix there is thus an issue with overflow/underflow

tammoippen commented 4 years ago

@audatic-henning Should be fixed in v3.7.2 . Can you please verify.

audatic-henning commented 4 years ago

Solved it for me, thanks a lot for the quick solution!