search

deephaven / deephaven-core

Deephaven Community Core
Other
257 stars 80 forks source link

`time_window` does not work in the embedded Python server #6405

Closed jjbrosnan closed 2 hours ago

jjbrosnan commented 1 day ago

Description

I've only tested this in JupyterLab, but it's likely the embedded Python server entirely:

Run the following code:

from deephaven.experimental import time_window
from deephaven import time_table

source = time_table("PT00:00:01")

result = time_window(source, "Timestamp", 5000000000, "WithinLastFiveSeconds")

Expected results

result to contain the last 5 seconds of data.

Actual results

An error with the following stack trace:

RuntimeError                              Traceback (most recent call last)
File /opt/conda/lib/python3.11/site-packages/deephaven/experimental/__init__.py:34, in time_window(table, ts_col, window, bool_col)
     33 try:
---> 34     return Table(j_table=_JWindowCheck.addTimeWindow(table.j_table, ts_col, window, bool_col))
     35 except Exception as e:

RuntimeError: java.lang.IllegalStateException: May not initiate serial table operations for update graph DEFAULT: exclusiveLockHeld=false, sharedLockHeld=false, currentThreadProcessesUpdates=false
    at io.deephaven.engine.updategraph.UpdateGraph.checkInitiateSerialTableOperation(UpdateGraph.java:184)
    at io.deephaven.engine.util.WindowCheck.addTimeWindowInternal(WindowCheck.java:98)
    at io.deephaven.engine.util.WindowCheck.lambda$addTimeWindow$0(WindowCheck.java:79)
    at io.deephaven.engine.table.impl.perf.QueryPerformanceRecorder.withNugget(QueryPerformanceRecorder.java:369)
    at io.deephaven.engine.util.WindowCheck.addTimeWindow(WindowCheck.java:77)

The above exception was the direct cause of the following exception:

DHError                                   Traceback (most recent call last)
Cell In[2], line 6
      2 from deephaven import time_table
      4 source = time_table("PT00:00:01")
----> 6 result = time_window(source, "Timestamp", 5000000000, "WithinLastFiveSeconds")
      8 display(DeephavenWidget(result))

File /opt/conda/lib/python3.11/site-packages/deephaven/experimental/__init__.py:36, in time_window(table, ts_col, window, bool_col)
     34     return Table(j_table=_JWindowCheck.addTimeWindow(table.j_table, ts_col, window, bool_col))
     35 except Exception as e:
---> 36     raise DHError(e, "failed to create a time window table.") from e

DHError: failed to create a time window table. : RuntimeError: java.lang.IllegalStateException: May not initiate serial table operations for update graph DEFAULT: exclusiveLockHeld=false, sharedLockHeld=false, currentThreadProcessesUpdates=false
Traceback (most recent call last):
  File "/opt/conda/lib/python3.11/site-packages/deephaven/experimental/__init__.py", line 34, in time_window
    return Table(j_table=_JWindowCheck.addTimeWindow(table.j_table, ts_col, window, bool_col))
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: java.lang.IllegalStateException: May not initiate serial table operations for update graph DEFAULT: exclusiveLockHeld=false, sharedLockHeld=false, currentThreadProcessesUpdates=false
    at io.deephaven.engine.updategraph.UpdateGraph.checkInitiateSerialTableOperation(UpdateGraph.java:184)
    at io.deephaven.engine.util.WindowCheck.addTimeWindowInternal(WindowCheck.java:98)
    at io.deephaven.engine.util.WindowCheck.lambda$addTimeWindow$0(WindowCheck.java:79)
    at io.deephaven.engine.table.impl.perf.QueryPerformanceRecorder.withNugget(QueryPerformanceRecorder.java:369)
    at io.deephaven.engine.util.WindowCheck.addTimeWindow(WindowCheck.java:77)

Additional details and attachments

This was reported and verified in JupyterLab. I have not tested this in the embedded server run from Python without Jupyter.

Versions

niloc132 commented 1 day ago

(was accidentally assigned to me, deliberately adding @jmao-denver to the ticket)

I think this just needs a call to update_graph.auto_locking_ctx in there to ensure we hold the lock if needed?