Timeout when exporting chart based on large dataset

[oscarsommerer]

Description

Calling Chart.toBase64PNGString (or any other export function) on a chart backed by a large enough dataset leads to a PuppeteerSharp timeout.

I'm trying to render a PointDensity chart with about one million items for the x/y values in a .NET interactive session (vscode polyglot notebook). Without exporting, this is done in a few seconds, but when trying to export the chart I get a timeout exception from PuppeteerSharp.

Repro steps

Run the F# following snippet

open Plotly.NET
open Plotly.NET.ImageExport

let x = [ for i in 0 .. 1000000 -> System.Random.Shared.Next() ]
let y = [ for i in 0 .. 1000000 -> System.Random.Shared.Next() ]

Chart.PointDensity(x = x, y = y)
|> Chart.toBase64PNGString (Width = 300, Height = 300)

Expected behavior

The export should return a base64 PNG string, even if generating the image takes longer than 30 seconds.

I'm wondering if it would be feasible to supply a custom timeout?

Actual behavior

The function throws a TimeoutException after about 30 seconds.

Error: System.AggregateException: One or more errors occurred. (One or more errors occurred. (Timeout of 30000 ms exceeded))
---> System.AggregateException: One or more errors occurred. (Timeout of 30000 ms exceeded)
---> System.TimeoutException: Timeout of 30000 ms exceeded
at PuppeteerSharp.Helpers.TaskHelper.<>c__DisplayClass2_0.<WithTimeout>b__0() in C:\projects\puppeteer-sharp\lib\PuppeteerSharp\Helpers\TaskHelper.cs:line 48
at PuppeteerSharp.Helpers.TaskHelper.WithTimeout(Task task, Func`1 timeoutAction, TimeSpan timeout, CancellationToken cancellationToken) in C:\projects\puppeteer-sharp\lib\PuppeteerSharp\Helpers\TaskHelper.cs:line 93
at PuppeteerSharp.DOMWorld.SetContentAsync(String html, NavigationOptions options) in C:\projects\puppeteer-sharp\lib\PuppeteerSharp\DOMWorld.cs:line 253
--- End of inner exception stack trace ---
at <StartupCode$Plotly-NET-ImageExport>.$PuppeteerSharpRenderer.clo@66-3.MoveNext()
at <StartupCode$Plotly-NET-ImageExport>.$PuppeteerSharpRenderer.Plotly-NET-ImageExport-IGenericChartRenderer-RenderPNGAsync@143.MoveNext()
--- End of inner exception stack trace ---
at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions)
at System.Threading.Tasks.Task`1.GetResultCore(Boolean waitCompletionNotification)
at Plotly.NET.ImageExport.AsyncHelper.taskSync@76.Invoke(Task`1 t)
at Plotly.NET.ImageExport.AsyncHelper.runSync[a,b](FSharpFunc`2 job, a input)
at Plotly.NET.ImageExport.AsyncHelper.taskSync[a](Task`1 task)
at Plotly.NET.ImageExport.ChartExtensions.ToBase64PNGString@134.Invoke(GenericChart gChart)
at <StartupCode$FSI_0012>.$FSI_0012.main@()
at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
at System.Reflection.MethodInvoker.Invoke(Object obj, IntPtr* args, BindingFlags invokeAttr)

Known workarounds

Rendering the chart without the static export function and using the export button works as expected.

Reducing the number of items to about one hundred thousand also seems to let PuppeteerSharp render the image fast enough, so as to not run into the timeout

Related information

• OS: Fedora 38
• Plotly.NET.ImageExport, 4.0.0
• Plotly.NET.Interactive, 4.1.0
• .NET Runtime 7.0.302
• Running a fairly old Intel® Core™ i7-3770 CPU

[kMutagene]

Have you tried setting the timeout on Plotly.NET.ImageExport.PuppeteerSharpRendererOptions.launchOptions ? Its weird though that your timeout occurs after 30 seconds, as the default is set to 60 seconds: https://github.com/plotly/Plotly.NET/blob/b9fc7cdc3fe27ed4a819fde6e73cebe711d2b8f4/src/Plotly.NET.ImageExport/PuppeteerSharpRenderer.fs#L14-L22

[oscarsommerer]

@kMutagene yep already tried that. Sadly this didn't change anything. I'm guessing that there is another timeout when doing puppeteer tasks somewhere? Inspecting my running processes clearly shows chrome running and taking a lot of CPU time

[oscarsommerer]

Having a look through the code it seems as this line in PuppeteerSharpRenderer.fs could be the offender? According to the Puppeteer docs it takes a NavigationOptions object which contains a timeout whose default value seems to be 30 seconds?

[kMutagene]

Thanks for the pointer, that looks promising. If you want to have a go at it, feel free to open a PR. Otherwise I opened #394 to track this.

[kMutagene]

You can access navigationOptions on PuppeteerSharpRendererOptions with the next release.

Your snippet took longer than 10 minutes, so i reduced the amount of points a bit for testing. This took around 1 min for me :

  let x = [ for i in 0 .. 500000 -> System.Random.Shared.Next() ]
  let y = [ for i in 0 .. 500000 -> System.Random.Shared.Next() ]

  PuppeteerSharpRendererOptions.navigationOptions.Timeout <- 0 // setting to 0 means no timeout

  Chart.PointDensity(x = x, y = y)
  |> Chart.toBase64PNGString (Width = 300, Height = 300)