search

posit-dev / positron

Positron, a next-generation data science IDE
https://positron.posit.co
Other
2.77k stars 84 forks source link

Attempting to return a polars lazyframe removes the console #4502

Open jthomasmock opened 2 months ago

jthomasmock commented 2 months ago

System details:

Positron and OS details: Positron Version: 2024.08.0 (Universal) build 48

Code - OSS Version: 1.91.0 Commit: ed616b36655fb24d116108bdd833f1321704315b Date: 2024-08-19T04:26:51.868Z Electron: 29.4.0 Chromium: 122.0.6261.156 Node.js: 20.9.0 V8: 12.2.281.27-electron.0 OS: Darwin arm64 23.5.0

Interpreter details:

Python 3.12.0

Describe the issue:

Attempting to print a lazy polars dataframe, ie, df.lazy() will remove the console.

https://github.com/user-attachments/assets/6e331326-87e9-443c-a396-5ecb0bed26ac

Steps to reproduce the issue:

  1. Create a lazy dataframe:
import polars as pl
df = pl.DataFrame({"a": [1, 2, 3], "b": [4, 5, 6]})
lf = df.lazy()
  1. Note that the console will disappear
  2. You can bring the console back by reloading the webview.

Expected or desired behavior:

The lazyframe should at least not remove the console, and if executed should show the query plan:

For example if you do print(lf), it will work fine:

print(lf)
naive plan: (run LazyFrame.explain(optimized=True) to see the optimized plan)

DF ["a", "b"]; PROJECT */2 COLUMNS; SELECTION: None

An example of calling lf would be something like:

<LazyFrame [2 operations]>
Operations:
    [1] filter [col("age") > 30]
    [2] select [columns = ['name', 'age']]

Were there any error messages in the UI, Output panel, or Developer Tools console?

In python console output:

Positron] SEND execute_request: OK
[Positron] Python kernel status changed: idle => busy
[Positron] RECV status/busy from iopub: {"header":{"msg_id":"5c83c4f8-aa828f65988030da9d331fc7_45638_118","msg_type":"status","username":"thomasmock","session":"5c83c4f8-aa828f65988030da9d331fc7","date":"2024-08-27T20:28:58.344230Z","version":"5.3"},"parent_header":{"msg_id":"fragment-666be1dd-8540-469b-96f6-82aa45b2983c","msg_type":"execute_request","version":"5.0","date":"2024-08-27T20:28:58.343000Z","session":"9a9525fc6fef51d2cabc1a4a5778cf5a","username":"thomasmock"},"metadata":{},"content":{"execution_state":"busy"},"buffers":[]}
[Positron] RECV execute_input from iopub: {"header":{"msg_id":"5c83c4f8-aa828f65988030da9d331fc7_45638_119","msg_type":"execute_input","username":"thomasmock","session":"5c83c4f8-aa828f65988030da9d331fc7","date":"2024-08-27T20:28:58.345060Z","version":"5.3"},"parent_header":{"msg_id":"fragment-666be1dd-8540-469b-96f6-82aa45b2983c","msg_type":"execute_request","version":"5.0","date":"2024-08-27T20:28:58.343000Z","session":"9a9525fc6fef51d2cabc1a4a5778cf5a","username":"thomasmock"},"metadata":{},"content":{"code":"lf","execution_count":5},"buffers":[]}
[Positron] RECV execute_result from iopub: {"header":{"msg_id":"5c83c4f8-aa828f65988030da9d331fc7_45638_120","msg_type":"execute_result","username":"thomasmock","session":"5c83c4f8-aa828f65988030da9d331fc7","date":"2024-08-27T20:28:58.347484Z","version":"5.3"},"parent_header":{"msg_id":"fragment-666be1dd-8540-469b-96f6-82aa45b2983c","msg_type":"execute_request","version":"5.0","date":"2024-08-27T20:28:58.343000Z","session":"9a9525fc6fef51d2cabc1a4a5778cf5a","username":"thomasmock"},"metadata":{},"content":{"data":{"text/plain":"<LazyFrame at 0x12308BB00>","text/html":"<h4>NAIVE QUERY PLAN</h4><p>run <b>LazyFrame.show_graph()</b> to see the optimized version</p><?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?>\n<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n<!-- Generated by graphviz version 10.0.1 (20240210.2158)\n -->\n<!-- Title: polars_query Pages: 1 -->\n<svg width=\"68pt\" height=\"66pt\"\n viewBox=\"0.00 0.00 67.50 65.50\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n<g id=\"graph0\" class=\"graph\" transform=\"scale(1 1) rotate(0) translate(4 61.5)\">\n<title>polars_query</title>\n<polygon fill=\"white\" stroke=\"none\" points=\"-4,4 -4,-61.5 63.5,-61.5 63.5,4 -4,4\"/>\n<!-- p1 -->\n<g id=\"node1\" class=\"node\">\n<title>p1</title>\n<polygon fill=\"none\" stroke=\"black\" points=\"59.5,-57.5 0,-57.5 0,0 59.5,0 59.5,-57.5\"/>\n<text text-anchor=\"middle\" x=\"29.75\" y=\"-40.2\" font-family=\"Times,serif\" font-size=\"14.00\">TABLE</text>\n<text text-anchor=\"middle\" x=\"29.75\" y=\"-23.7\" font-family=\"Times,serif\" font-size=\"14.00\">π */2;</text>\n<text text-anchor=\"middle\" x=\"29.75\" y=\"-7.2\" font-family=\"Times,serif\" font-size=\"14.00\">σ None</text>\n</g>\n</g>\n</svg>\n"},"metadata":{},"execution_count":5},"buffers":[]}
[Positron] RECV comm_msg/positron-variables-python-2-6d6e684e/undefined from iopub: {"header":{"msg_id":"5c83c4f8-aa828f65988030da9d331fc7_45638_121","msg_type":"comm_msg","username":"thomasmock","session":"5c83c4f8-aa828f65988030da9d331fc7","date":"2024-08-27T20:28:58.604378Z","version":"5.3"},"parent_header":{"msg_id":"fragment-666be1dd-8540-469b-96f6-82aa45b2983c","msg_type":"execute_request","version":"5.0","date":"2024-08-27T20:28:58.343000Z","session":"9a9525fc6fef51d2cabc1a4a5778cf5a","username":"thomasmock"},"metadata":{},"content":{"data":{"jsonrpc":"2.0","method":"update","params":{"assigned":[],"unevaluated":[{"access_key":"{\"type\":\"str\",\"data\":\"lf\"}","display_name":"lf","display_value":"<LazyFrame at 0x12308BB00>","display_type":"LazyFrame","type_info":"polars.lazyframe.frame.LazyFrame","size":48,"kind":"other","length":77,"has_children":true,"has_viewer":false,"is_truncated":false,"updated_time":1724790538604}],"removed":[],"version":0}},"comm_id":"positron-variables-python-2-6d6e684e"},"buffers":[]}
[Positron] Python kernel status changed: busy => idle
[Positron] RECV status/idle from iopub: {"header":{"msg_id":"5c83c4f8-aa828f65988030da9d331fc7_45638_123","msg_type":"status","username":"thomasmock","session":"5c83c4f8-aa828f65988030da9d331fc7","date":"2024-08-27T20:28:58.605807Z","version":"5.3"},"parent_header":{"msg_id":"fragment-666be1dd-8540-469b-96f6-82aa45b2983c","msg_type":"execute_request","version":"5.0","date":"2024-08-27T20:28:58.343000Z","session":"9a9525fc6fef51d2cabc1a4a5778cf5a","username":"thomasmock"},"metadata":{},"content":{"execution_state":"idle"},"buffers":[]}
[Positron] RECV execute_reply from shell: {"header":{"msg_id":"5c83c4f8-aa828f65988030da9d331fc7_45638_122","msg_type":"execute_reply","username":"thomasmock","session":"5c83c4f8-aa828f65988030da9d331fc7","date":"2024-08-27T20:28:58.605477Z","version":"5.3"},"parent_header":{"msg_id":"fragment-666be1dd-8540-469b-96f6-82aa45b2983c","msg_type":"execute_request","version":"5.0","date":"2024-08-27T20:28:58.343000Z","session":"9a9525fc6fef51d2cabc1a4a5778cf5a","username":"thomasmock"},"metadata":{"started":"2024-08-27T20:28:58.345025Z","dependencies_met":true,"engine":"9d5a8c8b-e767-4eff-af69-4c1047263838","status":"ok"},"content":{"status":"ok","execution_count":5,"user_expressions":{},"payload":[]},"buffers":[]}
[Positron] SEND heartbeat with timeout of 30 seconds
jmcphers commented 2 months ago 
"text/html":"<h4>NAIVE QUERY PLAN</h4>..."

This is probably what's crashing the Console -- not this specific bit of HTML, but something else in that string (maybe the XML bits?). This HTML is getting parsed for inline display and likely is causing a parsing exception, which leads to a React rendering exception, which leads to the React component not rendering.

The fix is either to figure out what the parser is choking on and address that, or promote content with this pattern to the Viewer pane instead.