All memory allocated by context not being freed

NullSoldier commented 3 years ago

Issue or Feature

When I allocate a context, there is seemingly nothing I can do to free the memory used. It seems to keep the memory without being freed. Also... If I clear the canvas and allocate a second one, it seems to re-use the buffer of the first canvas under certain conditions.

Creating context from a 10kx10k canvas allocated 400mb
Deleting context and canvas does not free memory
There are very strange allocation behaviour depending on the order you create the canvas

Other oddities are that reallocating multiple canvas and context does not use more memory seems to re-use memory pool if you allocate the canvas next to each other...

let canvas1 = createCanvas(10000, 10000)
let canvas2 = createCanvas(10000, 10000)

let context1 = canvas1.getContext('2d')
let context2 = canvas2.getContext('2d')
//    rss /    heap /     ext
// 413 MiB /   2 MiB /   1 MiB

but doubles memory when allocated interleaved

let canvas1 = createCanvas(10000, 10000)
let context1 = canvas1.getContext('2d')

let canvas2 = createCanvas(10000, 10000)
let context2 = canvas2.getContext('2d')
//    rss /    heap /     ext
//793 MiB /   2 MiB /   1 MiB / context created

Steps to Reproduce

const { createCanvas, Canvas, loadImage, Image } = require('canvas')

async function printMem(name) {
    await new Promise((r) => setTimeout(r, 1000))
    global.gc()

    const mem = process.memoryUsage()
    const rss = (mem.rss/1024/1024).toFixed(0).padStart(3)
    const heap = (mem.heapUsed/1024/1024).toFixed(0).padStart(3)
    const external = (mem.external/1024/1024).toFixed(0).padStart(3)
    console.log(`${rss} MiB / ${heap} MiB / ${external} MiB / ${name}`)
}

async function run() {
    console.log('    rss /    heap /     ext / name')
    await printMem('start')

    let canvas = createCanvas(10000, 10000)
    await printMem('canvas created')

    let context = canvas.getContext('2d')
    await printMem('context created')

    context = null
    await printMem('nulled context')

    canvas = null
    await printMem('nulled canvas all')
}

run()
//     rss /    heap /     ext / name
//  29 MiB /   2 MiB /   1 MiB / start
//  29 MiB /   2 MiB /   1 MiB / canvas created
// 413 MiB /   2 MiB /   1 MiB / context created
// 412 MiB /   2 MiB /   1 MiB / nulled context
// 412 MiB /   2 MiB /   1 MiB / nulled canvas all

Your Environment

canvas@2.6.1
v12.20.1 on Mac OS X 10.15.7

markovicdenis commented 3 years ago

In my case the context doesn't take as much memory, only 3MB... but sadly a bigger problem is the "loadImage" function never releases memory either.

I modified your test and looped through it 10 times, with additional loadImage call and a timeout... seems like it never releases the memory.

    rss /    heap /     ext / name
 31 MiB /   3 MiB /   0 MiB / start
 31 MiB /   3 MiB /   0 MiB / canvas created
 34 MiB /   3 MiB /   0 MiB / context created
111 MiB /   3 MiB /   0 MiB / nulled context
112 MiB /   3 MiB /   0 MiB / nulled canvas all
111 MiB /   3 MiB /   0 MiB / nulled img
112 MiB /   3 MiB /   0 MiB / waited 10 seconds
    rss /    heap /     ext / name
112 MiB /   3 MiB /   0 MiB / start
112 MiB /   3 MiB /   0 MiB / canvas created
112 MiB /   3 MiB /   0 MiB / context created
112 MiB /   3 MiB /   0 MiB / nulled context
112 MiB /   3 MiB /   0 MiB / nulled canvas all
112 MiB /   3 MiB /   0 MiB / nulled img
112 MiB /   3 MiB /   0 MiB / waited 10 seconds
... [9 iterations later]
    rss /    heap /     ext / name
112 MiB /   3 MiB /   0 MiB / start
112 MiB /   3 MiB /   0 MiB / canvas created
112 MiB /   3 MiB /   0 MiB / context created
112 MiB /   3 MiB /   0 MiB / nulled context
112 MiB /   3 MiB /   0 MiB / nulled canvas all
112 MiB /   3 MiB /   0 MiB / nulled img
112 MiB /   3 MiB /   0 MiB / waited 10 seconds

zbjornson commented 3 years ago

Thanks for the nice repro steps. I can't reproduce this on Windows or Linux however:

$ node --expose-gc ./1738.js 
    rss /    heap /     ext / name
 36 MiB /   3 MiB /   0 MiB / start
 37 MiB /   3 MiB /   0 MiB / canvas created
 37 MiB /   3 MiB /   0 MiB / context created
 37 MiB /   3 MiB /   0 MiB / nulled context
 37 MiB /   3 MiB /   0 MiB / nulled canvas all

I'm surprised to see 400 MB allocated when the context is created.

If this is truly unique to MacOS, then it's possible it has to do with the system memory allocator.

Any other tips for reproducing it?

jhuckaby commented 2 years ago

I'm almost positive this has to do with pango preloading system fonts. On macOS, for example, when you create the first canvas, pango locates every font installed on the system and preloads all of them into memory.

I wonder if there is a way to tell pango to look elsewhere for fonts (e.g. in an empty directory), and not preload all the system fonts.

chearon commented 2 years ago

IIRC there is one Pango FontMap per process, and it would be lazily created, so that could definitely explain the statistics in the OP. You should see that to varying degrees on every platform though. Here is the Windows code loading all system fonts when a PangoWin32FontMap is created.

macOS ships with fonts for many of the world's languages, and some fonts, like CJK or emoji, can be pretty huge. It could be that macOS simply has more fonts, or that it's parsing glyph paths unnecessarily.

I wonder if there is a way to tell pango to look elsewhere for fonts (e.g. in an empty directory), and not preload all the system fonts.

I think we should do our own font selection like browsers do, and ship Canvas with a small basic set of fonts.

jhuckaby commented 2 years ago

I think we should do our own font selection like browsers do, and ship Canvas with a small basic set of fonts.

I wholeheartedly agree! 👍🏻

Automattic / node-canvas