RangeError: Start offset undefined is outside the bounds of the buffer

[Seelengrab]

Hi,

when I run my notebook I get the following error:

Failed to process update! RangeError: Start offset undefined is outside the bounds of the buffer
    at new Uint8Array (<anonymous>)
    at PlutoConnection.js:126
    at async do_all (PlutoConnection.js:58)

I can share the notebook that produces this if that helps, but sadly not all files it depends on :/ Suffice it to say that it's two .csv files of 2.3G and 753M, of the form String,String,String,Int64,Int64,Int64,Int64,Int64 and DateTime, Float64, Int64.

[fonsp]

The notebook would be helpful!

[Seelengrab]

https://gist.github.com/Seelengrab/27b87837e6809c17e8c8821b5328a493

If I can help with anything else, please let me know.

The behaviour I observe is that every cell runs fine until the last one with the call to plot. That one just runs for a while, after which the error gets thrown. I don't think the cell ever terminates.

[Seelengrab]

The versions in the Project.toml are these:

  [537997a7] AbstractPlotting v0.12.14
  [336ed68f] CSV v0.7.7
  [a93c6f00] DataFrames v0.21.7
  [1313f7d8] DataFramesMeta v0.5.1
  [d0351b0e] InspectDR v0.3.9
  [3b7a836e] PGFPlots v3.3.3
  [8314cec4] PGFPlotsX v1.2.10
  [91a5bcdd] Plots v1.6.10
  [c3e4b0f8] Pluto v0.11.14
  [7f904dfe] PlutoUI v0.6.8
  [d330b81b] PyPlot v2.9.0
  [276b4fcb] WGLMakie v0.2.9

I'm aware that I don't use all of these packages, but I don't think it makes a difference.

[fonsp]

If you remove the plot(...) cell, does the problem disappear?

[Seelengrab]

I've commented the plot cell, it didn't show the error (although I didn't wait 10 minutes or something like that). I've got other notebooks where plotting like that was not a problem though 🤷‍♂️ Should I plot inside a function?

[Seelengrab]

Uncommenting and running the cell crashed the browser tab though. Reloading crashes the tab again 🤔

I'm running Vivaldi 3.4.2066.76, which is basically Chromium 86.0.4240.112.

Julia version:

Julia Version 1.5.2 Commit 539f3ce (2020-09-23 23:17 UTC) Platform Info: OS: Linux (x86_64-linux-gnu) CPU: Intel(R) Core(TM) i7-6600U CPU @ 2.60GHz WORD_SIZE: 64 LIBM: libopenlibm LLVM: libLLVM-9.0.1 (ORCJIT, skylake) Environment: JULIA_NUM_THREADS = 4

That Linux up there is kind of half a lie, it's WSL.

[fonsp]

Are you out of memory perhaps?

[Seelengrab]

Unlikely - I've got 20GB and the datasets combined are not even 4GB.

That does give me an idea though: The browser is (apparently) just a 32 bit application. Let me check with a 64 bit browser.

[Seelengrab]

Well, 64-bit Edge (which is also Chromium based) also dies, with an Out Of Memory 🤔 The dataset is definitely loaded in memory though, the julia process has a whopping 7GB memory footprint. Maybe something during plotting needs a lot more memory and just dies? I don't get any errors from any worker though, so I assume that's fine..?

Maybe I shouldn't plot all of the data at once 😅

[Seelengrab]

Hmm I've just caught the crash while looking at my ressource usage and it really does seem to be true Out Of Memory and getting killed due to that. That's unfortunate.

[fonsp]

Yes it sounds like you are plotting too much data at once, at least for the JS (plotly) or SVG (GR) based backends.

If you really need to plot all data, you could switch to the GR backend and force PNG output:

begin
using Plots
gr()
end

using PlutoUI

plot(...) |> PlutoUI.as_png

[Seelengrab]

I'm not sure if it's actually a problem of the data I try to plot - I'm loading the big dataset, but not doing anything with it. It's the smaller one I'm trying to plot. I'll try to plot the small one without loading the big dataset.

[Seelengrab]

Oooh live rendering copies the data to the browsers memory 😬 That's not good - I guess I'll have to forsake interactivity then :/