Monotonic XY business time plots do not render monotonically

[chipkent]

This bug is observed in my branch where I am working on calendars. I have no reason to believe that the same would not happen in main.

from deephaven import empty_table
from deephaven.plot import Figure
from deephaven.calendar import calendar

t = empty_table(100).update(["X=now() + HOUR", "Y=i*i"])

p1 = Figure().plot_xy("Test", t=t, x="X", y="Y").show()
p2 = Figure().plot_xy("Test", t=t, x="X", y="Y").x_axis(calendar=calendar("USNYSE")).show()

Problem 1: p1 (1) has a spike in the middle of the plot and (2) does not actually draw the plot for any amount of zoom.

Problem 2: p2 (3) does not draw the plot and (4) does not properly use the business calendar for the x-axis. Note that the dates are very far off.

[dsmmcken]

X in that formula results in the same timestamp for the whole X column does it not? Which would give you the vertical line shown in p1.

[mofojed]

@dsmmcken if I go to nanosecond resolution, it does indeed get differences:

However, I agree that formula still seems suspect, and it's probably dependent on the machine with how spaced apart the timestamps will be. @chipkent can you please verify the formula?

[chipkent]

The query had a bug.

Here is another one:

from deephaven import empty_table
from deephaven.plot import Figure
from deephaven.calendar import calendar

t = empty_table(100000).update(["X=now() + i*MINUTE", "Y=i"])

p1 = Figure().plot_xy("Test", t=t, x="X", y="Y").show()
p2 = Figure().plot_xy("Test", t=t, x="X", y="Y").x_axis(calendar=calendar("USNYSE")).show()

p1 looks ok.

p2 still looks broken.

[chipkent]

I just reran the test on my branch, and at this instant, there does not appear to be a problem.

[chipkent]

Reproduced with:

from deephaven import empty_table
from deephaven.plot import Figure
from deephaven.calendar import calendar

t = empty_table(100000).update(["X='2023-11-14T00:00 ET' + i*MINUTE", "Y=i"])

p1 = Figure().plot_xy("Test", t=t, x="X", y="Y").show()
p2 = Figure().plot_xy("Test", t=t, x="X", y="Y").x_axis(calendar=calendar("USNYSE")).show()

[chipkent]

I'm digging more into this. I believe the problem is in the plot rendering code.

from deephaven import empty_table
from deephaven.plot import Figure
from deephaven.calendar import calendar
from deephaven.time import to_j_instant

start = to_j_instant('2023-11-14T00:00 ET')
t = empty_table(100000).update(["X=start + i*MINUTE", "Y=i"])

p1 = Figure().plot_xy("Test", t=t, x="X", y="Y").show()
p2 = Figure().plot_xy("Test", t=t, x="X", y="Y").x_axis(calendar=calendar("USNYSE")).show()

t2 = t.update(["Z = diffBusinessNanos(start,X)", "Bad = Z - Z_[i-1]"])

t3 = t2.where("!isNull(Bad) && Bad < 0")

import jpy

AxisTransformBusinessCalendar = jpy.get_type("io.deephaven.plot.axistransformations.AxisTransformBusinessCalendar")

atbc = AxisTransformBusinessCalendar(calendar("USNYSE"))

t4 = t2.update([
    "II=ii", 
    "XEpoch = epochNanos(X)",
    "XEpochD = (double)XEpoch",
    "T = atbc.transform(XEpochD)", 
    "IT = atbc.inverseTransform(T)",
    "ITInstant = epochNanosToInstant((long)IT)",
    "Delta = ITInstant - ITInstant_[ii-1]",
    ])

t5 = t4.where("!isNull(Delta) && Delta < 0")

p_dbg = Figure().plot_xy("ATBC", t=t4, x="II", y="T").show()
p_dbg2 = Figure().plot_xy("IN", t=t4, x="II", y="XEpochD").plot_xy("OUT", t=t4, x="II", y="IT").show()

The plot is broken:

The transformed values are monotonically increasing, as expected:

The inverse appears to also be monotonic:

[mofojed]

Just showing a screenshot where the hovered value shows Nov 24, but it's appearing closer to Dec 3 on the axis for some reason:

[mofojed]

Codepen to mess around with it: https://codepen.io/mofojed/pen/ZEwMagL

[mofojed]

Looks like there is a bug in plotly opened around this: https://github.com/plotly/plotly.js/issues/5783 Seems to only happen on specific dates. Will likely need to be fixed in Plotly upstream.