surface() with CairoMakie backend, lots of allocations?

[paulmelis]

using GeoMakie, JLD
using CairoMakie
#using GLMakie

function go()

    data = load("data.jld")

    for key in keys(data)
        println("$(key) $(typeof(data[key])) $(size(data[key]))")
    end

    lats = data["lats"]
    lons = data["lons"]
    values = data["values"]

    t0 = time()

    println("Figure()")    
    @time fig = Figure()

    println("GeoAxis()")
    @time ax = GeoAxis(fig[1,1],
               source="+proj=longlat +datum=WGS84",
               dest="+proj=lcc +lon_0=5 +lat_1=50 +lat_2=55")

    println("surface!()")
    @time surf = surface!(ax, lons, lats, values; shading=NoShading)

    println("Colorbar()")
    @time Colorbar(fig[2,1], surf; ticklabelsize=10, tickwidth=1, vertical=false)

    println("display()")
    @time display(fig)

    t1 = time()
    println("time-to-plot $((t1-t0)*1000)ms")
end

go()

First run from REPL (10.5s to plot):

melis@blackbox 12:59:~/projects/harmonie$ j --project 
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.10.4 (2024-06-04)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

julia> versioninfo()
Julia Version 1.10.4
Commit 48d4fd48430 (2024-06-04 10:41 UTC)
Build Info:
  Official https://julialang.org/ release
Platform Info:
  OS: Linux (x86_64-linux-gnu)
  CPU: 8 × Intel(R) Core(TM) i7-6700K CPU @ 4.00GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-15.0.7 (ORCJIT, skylake)
Threads: 1 default, 0 interactive, 1 GC (on 8 virtual cores)

julia> include("t_perf.jl")
[ Info: Precompiling GeoMakie [db073c08-6b98-4ee5-b6a4-5efafb3259c6]
[ Info: Skipping precompilation since __precompile__(false). Importing GeoMakie [db073c08-6b98-4ee5-b6a4-5efafb3259c6].
values Matrix{Float64} (390, 390)
lats Vector{Float64} (390,)
lons Vector{Float64} (390,)
Figure()
  0.012670 seconds (2.96 k allocations: 189.305 KiB, 91.84% compilation time)
GeoAxis()
  3.581337 seconds (5.73 M allocations: 391.191 MiB, 3.37% gc time, 99.44% compilation time: 2% of which was recompilation)
surface!()
  0.333464 seconds (490.03 k allocations: 34.229 MiB, 4.94% gc time, 99.30% compilation time)
Colorbar()
  3.206779 seconds (5.99 M allocations: 412.315 MiB, 2.66% gc time, 99.42% compilation time: 2% of which was recompilation)
display()
  3.372642 seconds (6.08 M allocations: 433.822 MiB, 8.75% gc time, 35.99% compilation time: 2% of which was recompilation)
time-to-plot 10509.93800163269ms

Second run, same REPL session (1.95s):

julia> include("t_perf.jl")
values Matrix{Float64} (390, 390)
lats Vector{Float64} (390,)
lons Vector{Float64} (390,)
Figure()
  0.000260 seconds (1.31 k allocations: 78.836 KiB)
GeoAxis()
  0.006039 seconds (22.18 k allocations: 3.159 MiB)
surface!()
  0.001010 seconds (549 allocations: 637.062 KiB)
Colorbar()
  0.006804 seconds (22.04 k allocations: 2.229 MiB)
display()
  1.932373 seconds (3.86 M allocations: 284.856 MiB, 1.35% gc time)
time-to-plot 1946.8019008636475ms

Even in the second run the display() call seems to have lots of allocations for such a small dataset. Is this to be expected, and particular to GeoMakie?

[asinghvi17]

This could potentially be sped up by adding some precompile statements, I'll give that a look. The allocations when using CairoMakie are expected, because it internally translates the surface grid to a triangle mesh, which has 2*nx*ny faces.

[asinghvi17]

~Display is somehow taking 100 seconds for me on first run 😅 which is definitely unexpected...~

Edit: it turns out that I mixed up lons and lats when making this example work without the JLD file (which I didn't see in the issue), but that did expose an interesting bug in CairoMakie so I'll call it fair :D

[asinghvi17]

Here's a more minimal MWE, and the timings broadly agree with what you show:

using GeoMakie
using CairoMakie
#using GLMakie

function go()
    lons = LinRange(-180, 180, 390)
    lats = LinRange(-90, 90, 390)
    values = rand(length(lons), length(lats))

    t0 = time()

    println("Figure()")    
    @time fig = Figure()

    println("GeoAxis()")
    @time ax = GeoAxis(fig[1,1],
               source="+proj=longlat +datum=WGS84",
               dest="+proj=lcc +lon_0=5 +lat_1=50 +lat_2=55")

    println("surface!()")
    @time surf = surface!(ax, lons, lats, values; shading=NoShading)

    println("Colorbar()")
    @time Colorbar(fig[2,1], surf; ticklabelsize=10, tickwidth=1, vertical=false)

    println("display()")
    @time display(fig)

    t1 = time()
    println("time-to-plot $((t1-t0)*1000)ms")
end

go()

[asinghvi17]

Using TimerOutputs (manually inserted into Makie) gets me the result below, which clearly indicates that defining and drawing the mesh to Cairo takes the bulk of time here. We could get this down a bit maybe by implementing a mesh renderer in CairoMakie but that would take a while.

 ──────────────────────────────────────────────────────────────────────────────────────────
                                                  Time                    Allocations      
                                         ───────────────────────   ────────────────────────
            Tot / % measured:                 2.86s /  57.6%            302MiB /  89.6%    

 Section                         ncalls     time    %tot     avg     alloc    %tot      avg
 ──────────────────────────────────────────────────────────────────────────────────────────
 surface                              1    1.64s   99.5%   1.64s    269MiB   99.4%   269MiB
   draw_mesh3D                        1    1.63s   99.0%   1.63s    249MiB   91.9%   249MiB
     draw_mesh3D_decomposed           1    1.40s   85.1%   1.40s    153MiB   56.6%   153MiB
       draw_pattern                   1    1.31s   79.4%   1.31s   39.2MiB   14.4%  39.2MiB
         Cairo drawing             302k    1.21s   73.6%  4.02μs   4.61MiB    1.7%    16.0B
       apply transform                1   21.7ms    1.3%  21.7ms   2.32MiB    0.9%  2.32MiB
       camera to screen space         1    590μs    0.0%   590μs   1.74MiB    0.6%  1.74MiB
     per face colors                  1    228ms   13.8%   228ms   90.5MiB   33.4%  90.5MiB
     mesh attribute extraction        1   1.15ms    0.1%  1.15ms   5.20MiB    1.9%  5.20MiB
   Surface2Mesh                       1   8.31ms    0.5%  8.31ms   20.3MiB    7.5%  20.3MiB
   color extraction                   1   12.6μs    0.0%  12.6μs      224B    0.0%     224B
   faceculling                        1   7.33μs    0.0%  7.33μs      576B    0.0%     576B
 lines                                3   6.64ms    0.4%  2.21ms   1.49MiB    0.5%   508KiB
 text                                 6    513μs    0.0%  85.5μs   57.8KiB    0.0%  9.63KiB
 image                                1    252μs    0.0%   252μs   6.67KiB    0.0%  6.67KiB
 linesegments                         7   89.5μs    0.0%  12.8μs   8.09KiB    0.0%  1.16KiB
 ──────────────────────────────────────────────────────────────────────────────────────────

[paulmelis]

The allocations when using CairoMakie are expected, because it internally translates the surface grid to a triangle mesh, which has 2*nx*ny faces.

Why draw as a triangle mesh, and not quads? That would potentially half the number of draw calls (and I'm sure Cairo can support it)

[asinghvi17]

Quads are not well supported supported by vector graphics (Cairo usually rasterizes them in many formats, and the file sizes get pretty insane even if not rasterized), and the majority of time is spent preparing the Cairo pattern anyway.

I had tried this before but the tradeoff simply isn't worth it - the better option is to implement a pure Julia mesh rasterizer (not actually too hard) and then rasterize the mesh to an image, and emit the image.

[paulmelis]

Quads are not well supported supported by vector graphics (Cairo usually rasterizes them in many formats, and the file sizes get pretty insane even if not rasterized)

I don't know all the details of what frequently gets output from Cairo when used from Julia (I only use it to rasterize vector graphics directly from C/C++, not to save to svg for example), but to which formats does it rasterize?

and the majority of time is spent preparing the Cairo pattern anyway.

Is that in https://github.com/MakieOrg/Makie.jl/blob/77f2cfdb15b85374f449c5cdc3ad3c108cf858e2/CairoMakie/src/primitives.jl#L975? Can't you create the pattern once and then reuse it for all triangles?

[asinghvi17]

output from Cairo when used from Julia

Usually SVG (quads not supported, so the whole SVG gets rasterized internally), PDF (quads supported but filesize >100mb vs 20mb with triangles), or PNG (long rasterization time).

Can't you create the pattern once and then reuse it for all triangles

Not to my knowledge, unless Cairo has a nice API for that? We are changing the points and the colors at each pattern, so it's a nontrivial reconstruction.

[asinghvi17]

cf https://github.com/MakieOrg/Makie.jl/issues/4112 - this may be the reason why this issue exists.