vaexio / vaex

Out-of-Core hybrid Apache Arrow/NumPy DataFrame for Python, ML, visualization and exploration of big tabular data at a billion rows per second 🚀
https://vaex.io
MIT License
8.27k stars 590 forks source link

DataFrame.max returning array containing -inf values #2386

Open RobbieTClark opened 1 year ago

RobbieTClark commented 1 year ago

Hi, I have a large time series dataset in which I am using vaex for the backend of a graph to dynamically downsample/upsample when zooming in and out. I am trying to use the DataFrame.max function to bin the values in regular intervals so that the maximum of each window/block of data that is skipped over during downsampling is visible (Instead of the downsampling via 'every n samples').

The problem arises when using the max function as the array that is returned contains -inf values and I cannot understand why. Any ideas?

If I decrease the number of points that are sampled using the shape arg it seems to reduce the number of -inf values that are returned relative to the array size.