High point count `Chart.Line3D` "crashes" a Polyglot Notebook

Description

Inside a Polyglot Notebook with F# and Markdown cells, Chart.Line3D with 700,000 points works fine, albeit slow. A Chart.Line3D with 1,400,000 points "crashes" the notebook in the sense that the background turns gray, all outputs for all cells disappear, and cells cannot be re-ran and outputs displayed. To recover, "Clear All Outputs" must be selected, and then the VS Code window must be reloaded or VS Code closed and re-opened. I am writing "crash" in quotes because I don't know exactly what is crashing or failing. It is unclear whether this is a Polyglot Notebook, Plotly.NET, or Plotly issue, so I am starting here.

Plot with 700,000 points:

Plot with 1,400,000 points:

As an ancillary concern, the plotting is quite slow. F# generates the 700,000 points in less than 0.1s, while plotting just a 2D plot of one of the axis takes around 4-5s. So it takes approximately 40-50x longer to plot than it does to generate the data, even for a 2D plot. Is this expected?

Repro steps

Run the attached notebook. Notice that the first 3D Rossler plot works with no problem. However, once the rosslerHighIterations plot is attempted, the notebook "crashes". It is best to run the cells one at a time to see this behavior clearly in terms of which cell causes the "crash".

Expected behavior

The 3D plot is displayed.

Actual behavior

The notebook's background turns from black to gray, all outputs for all cells disappear, and the notebook becomes unresponsive to cells running with no outputs displayed. The notebook must have its outputs cleared and then VS Code re-opened.

Known workarounds

Reduce point count, primarily by reducing step size. I did try using Chart.show to save the plot to a temporary HTML file and then view it in the browser. That works but still not for the 1,400,000 point count plot. When the chart displays both in a notebook and a browser via Chart.show, I do see that Chart.show is much faster (about 2-3x faster) to get to a plot.

Related information

Operating system: Windows 11
Branch: I don't know how to answer that. I am using #r "nuget: Plotly.NET.Interactive, 4.1.0" as noted from the Plot.NET website. I also tried just using #r "nuget: Plotly.NET.Interactive to get the latest version, which chose version 4.2.1. The issue is the same.
.NET Runtime, CoreCLR or Mono Version: I don't know. Polyglot Notebooks bundle their own .NET and F# versions, to my knowledge.
Performance information, links to performance testing scripts

chaos-and-fractals.zip

Sadly, this is to be expected on this amount of data.

A little technical background: Both the output in interactive notebooks and Chart.show pages are just html that contains the necessary javascript to render a plotly chart. The F# side is 'only' constructing a json string in a type safe manner that is then passed to plotly.js in this html scaffold, that's why F# is so 'fast' here.

So what you are experiencing is a limitation of the rendering engine of plotly.js (which is basically D3.js). Don't forget that it is not drawing a static image, but all these points must be rendered in a way to allow hovering interactions, zooming, scrolling etc. The reason for your browser rendering the plots faster could be a limitation on how vscode renders html or executes javascript in notebooks.

What can you do?

if you do not need interactivity: render the chart as a static image. You can either save the images locally and include them in the notebook, or include them via base64 encoding in the notebook output. An example can be seen here. This might be slow, but will not freeze your notebook once the image is rendered.
for 2D charts: enable UseWebGL option, this is a lot more responsive for large amounts of data.
other charts: you seem to be out-of luck here. The only thing i can think of is what you already posted - decreasing the amount of individual datums plotted.

Ah, one thing i forgot: you can also use the chart config to let plotly.js render a static plot.

I suspect the freeze is coming from the interactivity of the chart. so if you have no problem waiting a bit for the chart to get rendered you can set it as static plot via the config. I at least had no problem rendering 10M points on a 2d graph without any notebook freezes.

Sadly, this setting will not work for 3D plots, it is a known issue from plotly.js.

Thank you for all the helpful information!

I was worried that this was likely a limitation of Plotly itself, but I wanted to check. A couple follow-up questions: Is there anything we can do to keep Plotly from crashing the notebook in terms of how Plotly.NET.Interactive interacts with the notebook's output cell? Some sort of "hey, this is a lot of \<stuff>, it ain't going to work" or somehow detecting Plotly has gone wrong? I haven't looked at that interaction, and it's been a while before I last looked into how the custom display "kernels" worked in Polyglot Notebooks. Or is that really on the side of the Polyglot Notebook implementation and/or VS Code? If there's something I can file for the Polyglot Notebook team to take a look at, as this might be more on their side of things, I don't mind doing so.

Thank you for the heads up on WebGL mode for the 2D plots. I'll try that out.

For this particular use case, I've just been rolling with bigger step sizes to get lower point counts. I don't really need the 1,400,000 point resolution, lol, as it's just what the book I'm following used in their render and what I happened to try first. If I need a high-resolution render, I can try out the static image output you suggested. For the normal case, rotating these attractors around in 3D is half the fun, haha, so I'll just work with the lower step size, which is actually perfectly fine in this case, to keep the interactivity.

Last question! I've used Polyglot Notebooks off and on for a bit now, but now I'm really diving into them a bit more headlong where I'm wanting to do more custom plots and style the plots a bit more exactly. Where is the best place to ask questions about Plotly.NET? I don't mind filing issues for things that are halfway between a question and possible bug report, and closing them agressively in the case of questions, but I wanted to ask if there was a go to spot for asking Plotly.NET questions.

Thanks again!

Is there anything we can do to keep Plotly from crashing the notebook in terms of how Plotly.NET.Interactive interacts with the notebook's output cell? [...] Or is that really on the side of the Polyglot Notebook implementation and/or VS Code?

I think in this case it is neither plotly or the polyglot notebook extension. At the point where the chart html is written as output, it is up to the VSCode notebook rendering mechanism to display the html (and execute the javascript in it), which in the end boils down to VSCode being an electron app. It seems to me like it is just too much to display/render for an app like this. I think that you are seeing slow, but non-crashing rendering in a browser window supports this assumption. We could limit the string size of the output html, however how can we ever determine if the thing is going to render or not? it might be that there is a lot of color-styling related stuff in there, which makes the json big, but renderable. I think this also depends on the individual machine it is run on. So I don't think there is much we can do here reliably.

Where is the best place to ask questions about Plotly.NET? I don't mind filing issues for things that are halfway between a question and possible bug report, and closing them aggressively in the case of questions, but I wanted to ask if there was a go to spot for asking Plotly.NET questions.

Currently, it is still the fslab discord server, however I start to dislike the 'information silo' nature of discord so we might soon move to github discussions. Issues are fine for the exact case you mentioned, where it is a little more than just a question (just like this very issue here).

We could limit the string size of the output html, however how can we ever determine if the thing is going to render or not?

Yea, this is what I was thinking as well. It's probably really hard, if not impossible to always get something like that right with all the permutations available, so it does make sense to just let it ride and see if it renders. Haha! Thank you for the information though on why this is more of a VS Code / Electron thing rather than anything to do with Polyglot Notebooks.

Currently, it is still the fslab discord server, however I start to dislike the 'information silo' nature of discord so we might soon move to github discussions.

I don't personally manage any open-source projects, so this is from a contributor/user/etc. perspective, but I've always liked GitHub Discussions since they are just that and don't pollute Issues with questions. And it's nice to link to them from Issues when a Discussion uncovers an issue.

I'll drop a couple questions in the Discord, as I'm having some issues understanding getting what I'd like, mainly for grid subplots. Thanks for pointing me there.

Thanks again for all the information and for Plotly.NET! Slowly getting the hang of things and generating some sweet plots.

plotly / Plotly.NET