Plotting null values results in a constantly-spinning wheel

[jjbrosnan]

Description

Working with one of our users, he mentioned that he was unable to plot the null data he has in a table. I've been able to reproduce his issue not only trying to plot more than one column (both in and outside of a loop), but just trying to create a single plot.

Steps to reproduce

Message me about the data in Parquet that will repro this issue (GitHub won't let me attach even a small portion of it). I'll send it to you. Once you've got it, stick it in the data directory of your DHC installation.

Then, from the DH server side Python API:

from deephaven.parquet import read as read_pq
from deephaven.plot.figure import Figure

t = read_pq("/data/WontPlot.parquet")

fig = Figure().plot_xy(series_name="WontPlot", t=t, x="timestamp", y="Signal11").show()

Expected results

The plot to display with the data.

Actual results

A plot to display, but it's empty and has a forever-spinning wheel. Also, the browser logs show a null pointer exception.

Additional details and attachments

2023-06-01-151146_support_logs.zip

Versions

• Engine Version: 0.24.2
• Web UI Version:
• Java Version:
• Barrage Version:
• OS: OS X
• Browser: Chrome
• Docker: 20.10.13

[mofojed]

Another plot with null's from generated data that is failing:

from deephaven import empty_table
from deephaven.plot.figure import Figure
from deephaven.time import millis_to_datetime

t = empty_table(400000).update(["Timestamp=(DateTime)(millis_to_datetime(i))","x=(double)i", "y=i%12==11?null:Math.sin(i)"])
f = Figure().plot_xy(series_name="Hello", t=t, x="Timestamp", y="y").show()

Unsure if it is the same root cause.

[danilkryuchkov]

I was able to replicate (partly) the issue by the following: 1.

d = {'time': [1, 1, 2, 2, 2, 3, 3, 4, 4, 4],\
     'name': ['s1', 's2', 's1', 's2', 's3', 's1', 's3', 's1', 's2', 's3'],\
     'value': [1.7, 3.7, 2.7, 4.7, 4.7, 2.7, 2.7, 1.7, 6.7, 0.7]}

df = pd.DataFrame(data = d)

resulting in a df that looks like this (no empty/null cells):

2. The we do a pivot() (which was also the way null data that opened this issue was created) resulting in the following dataframe: df = df.pivot(index='time', columns='name', values='value').reset_index()

Note: empty cells in pandas dataframe actually contain numpy.nan in them.

2.1 (I tried both implementing and not implementing this step, result doesn't seem to change at all) We can try running df = df.replace(np.nan, None) The resulting dataframe:

Note: the Jupyter Notebook's "Data viewer" seems to display the contents of the cells slightly differently ,so something is changed, but later down the line (in Deephaven) this seems to make no difference.

3. We save the dataframe to parquet using pandas native df.to_parquet(path= "some path")

4. Now in Deephaven, we do sample_pivot_resetIndex = read('data/sample_pivot_resetIndex.parquet') And get a table that looks like this:

Note: now, the column types are as follows: time(long), s1(double), s2(double),s3(double). The "Copy Cell Unformatted" of one of the empty cells returns 'null' (not NaN or nan).

5. Plotting.

f = Figure()
plot1 = f.plot_xy(series_name='s1', t=sample_pivot_resetIndex, x='time', y='s1').show()

creates a plot that loads infinitely (despite both columns not having any null values)

Same goes for plotting a column that does have nulls plot1 = f.plot_xy(series_name='s2', t=sample_pivot_resetIndex, x='time', y='s2').show()

Finally, plot1 = f.plot_xy(series_name='s2', t=sample_pivot_resetIndex, x='s1', y='s2').show() plotting two (double) columns would not work either

6. Odd behavior. a) (it was initially odd that 'time' column had 'long' type in Deephaven instead of 'int') So if we do sample_pivot_resetIndex_int = sample_pivot_resetIndex.update(formulas=["time = (int)time"]) Now the column 'time' has type (int), and the rest remain (double) now we can do: plot1 = f.plot_xy(series_name='s1', t=sample_pivot_resetIndex_int, x='time', y='s1').show() and it plots just fine

and we can even do plot1 = f.plot_xy(series_name='s2', t=sample_pivot_resetIndex_int, x='time', y='s2').show()

to successfully plot with null values so it seems that in this case the nulls were not the issue, simply having that (long) column was. (even if (long) column was not a part of the plot at all (plotting s1 against s2) it still would not let the plot be created)

b) We have the original sample_pivot_resetIndex table with (long) column. running sample_pivot_resetIndex = sample_pivot_resetIndex.update(formulas=["time = replaceIfNull(time, NULL_DOUBLE)"]) ends up changing the 'time' column type to (double). I'm not sure at all why it does that. And the table then can be used for plotting, like in the previous example. (I found that out when running replaceIfNull(column_name, NULL_DOUBLE) on all columns hoping to see some change in 's2', or 's3' columns, but instead it just type casted the (double) type onto the 'time' column)

---

Going back to the original table that this issue was started from. (WontPlot.parquet) That table was also derived the same way (from a dataframe pivot()) the only real difference is that the timestamp column is of type (DateTime), and it is just way bigger. I also checked, and timestamp column does not contain any nulls/empty cells. I tried plotting timestamp with Signal1, and it kept having the same infinite load error. However, simply doing t = t.head(100000) and trying to plot again, resolved the issue and plotted sucessfully. (there are still null cells in the first 100,000 rows, so it still plots nulls just fine) So it seems like in that specific case, it was just the size of the table causing the issue.

I found that at head(370000) is still plottable , but at head(380000) it would go into infinite load.

Finally, I tried doing: t_small = t.select(formulas=['timestamp', 'Signal1']) And it would do the same thing where it would only plot up to .head(370000) and fail at .head(380000) (as expected) However, when I tried t_small = t.select(formulas=['timestamp', 'Signal1']).where(filters=["!isNull(Signal1)"]) It returned table with same 2 columns, but with 558000 rows (more than could previously plot, but without any null cells) and it was able to be plotted with no issues.

Overall, there seem to be a number of things that can cause a plot to fail, but I am not sure at all exactly what they are.

My final hypothesis about the WontPlot.parquet is that plotting with nulls is WAY more computationally expensive, so way more points can be plotted if no nulls are present before it begins to crash.

Hopefully this gave some useful examples/insights into what kinds of things I ran into. Please feel free to reach out about any details/clarifications.

[danilkryuchkov]

P.S. the Deephaven container that all of the above was done on had os.cpu_count() of 40 and 200GB of available RAM, so I dont think I was simply running out of hardware resources.

[mattrunyon]

Putting some updated code snippets and observations here

This works (but is slow and actually downsamples to 105k points)

from deephaven import empty_table
from deephaven.plot.figure import Figure

t = empty_table(400000).update(["Timestamp=epochMillisToInstant(i)","x=(double)i", "y=i%12==11?null:Math.sin(i)"])
f = Figure().plot_xy(series_name="Hello", t=t, x="Timestamp", y="y").show()

This does not work. Throws a class cast exception in the JS API and then has a failed assertion

from deephaven import empty_table
from deephaven.plot.figure import Figure

t = empty_table(4000000).update(["Timestamp=epochMillisToInstant(i)","x=(double)i", "y=i%12==11?null:Math.sin(i)"])
f = Figure().plot_xy(series_name="Hello", t=t, x="Timestamp", y="y").show()

Without nulls, a 40M row table downsamples to 7.3k pts. So something is up w/ downsampling when nulls are involved

from deephaven import empty_table
from deephaven.plot.figure import Figure

t = empty_table(40000000).update(["Timestamp=epochMillisToInstant(i)","x=(double)i", "y=Math.sin(i)"])
f = Figure().plot_xy(series_name="Hello", t=t, x="Timestamp", y="y").show()

I doubled the first example and got almost double the resulting points from the server. So seems to be a downsampling issue coupled with when the payload size explodes on the client an assertion fails and we never get notice that the downsampling failed.

[dsmmcken]

non consecutive nulls have to be preserved when down sampling, so each null will be present in the output I think.

[dsmmcken]

@niloc132 might be better to provide context.

[mattrunyon]

Related issues. One deals w/ the infinite load where >100MB payload is getting split and not reconciled on the client, so no event ever tells us downsampling is finished or failed. The other would be a nice feature so we don't end up getting a million points to plot from the server which would be horribly slow with plotly (and kind of defeats the purpose of downsampling).

https://github.com/deephaven/deephaven-core/issues/4563 https://github.com/deephaven/deephaven-core/issues/4564

[mattrunyon]

Since the issue needs some engine fixes, I'm backlogging this for now as there's nothing else we can do on the web UI side until those fixes are done