SVG export under node.js with paper-jsdom-canvas very slow

[mvanga]

I'm trying to generate some images (using paper-jsdom-canvas) that have a lot of different elements (a mix of circles, rectangles, and paths totalling around ~15k objects).

However, exporting this as an SVG image is turning out to be extremely slow: approximately between 40 and 50 minutes for a single export. Now, I understand that 15k objects is a lot, but I find it hard to believe that it would take over 40 minutes on a modern CPU to serialize that many objects. I've tried some optimizations like lowering the precision of the object positions but the speedup has been negligible. The biggest improvement came from removing the paper.view.update() call which brought times down to around ~25 minutes on my laptop.

My guess is that there's a ton of abstraction overhead kicking in due to the way paper.js (or perhaps the underlying jsdom representation) is being serialized. I've attached a simplified version of the code I'm using for anyone interested to reproduce this issue.

Does anyone have experience with speeding this up? If not, can anyone help me track this down (I'm fairly new to paper.js)? Any suggestions welcome!

var fs = require('fs');
var path = require('path');
var paper = require('paper-jsdom-canvas');

function random_range(min, max) {
  return Math.random() * (max - min) + min;
}

paper.setup(new paper.Size(500, 500));

for (var i = 0; i < 15000; i++) {
    console.log('iteration: ' + (i + 1));
    var cx = random_range(0, paper.view.size.width);
    var cy = random_range(0, paper.view.size.height);
    var r = random_range(3, 100);
    var c = "#000000".replace(/0/g,function(){return (~~(Math.random()*16)).toString(16);});
    var circle = new paper.Path.Circle({
        center: [cx, cy],
        radius: r,
        fillColor: c
    });
}

console.log('generating SVG');
var svg = paper.project.exportSVG({
    asString: true,
    precision: 2,
    matchShapes: true,
    embedImages: false
});

fs.writeFile(path.resolve('./out.svg'),svg, function (err) {
    if (err) throw err;
    console.log('Saved!');
});

[iconexperience]

A while ago I played with complex items and noticed the slow performance of SVG export for larger item trees. As far as I remember this was caused by the way attributes are collected in SVG export. For each attribute in each item the parent item is checked and the attribute is only set if it is different from the parent's attribute. This makes a lot of sense because it keeps the SVG file small.

I do not fully remember, but I think the problem was that the attributes for the parent were calculated over and over again for each child or something similar. I have to check again what really happens in detail.

A starting point is https://github.com/paperjs/paper.js/blob/develop/src/svg/SvgExport.js#L289

[iconexperience]

I tried to get a better understanding of what it happening. Here is what I think is going on:

• For each SVG attribute of the SVGExport gets each item's parent's value. If the item's value is different from the parent's value, it is set to the SVG element, otherwise it is dismissed, because the parent's value can be inherited.
• Each time SVGExport calls getXX() for a a parent's attribute, the parent iterates through its children to see if all have the same attribute value. If they do have the same value, it will actually iterate over all its children.

This means that if you have 50,000 children there will be 50,000 calls to the parent element for each attribute. This call then results in up to 50,000 calls to the parent's children. Consequently you will have 50,000 * 50,000 = 2.5 billion get() calls for each attribute.

This would explain why the export takes 40 minutes.

But please note that I only took a superficial look so far and might as well be wrong.

[mvanga]

Hey @iconexperience, thanks for the earlier tip and the second comment with your analysis! So basically the SVGExport function has n^2 complexity on the number of children.

So I get why the first (up)call is made to the parent for each of the children; it optimizes away duplicate information from the final SVG output. However, I guess I don't understand why the second (down)call is made. That is, why does the parent need to iterate over all its children?

Thanks a lot for the help!

[iconexperience]

@mvanga I think the second call is only needed because the parent's attributes are not cached. When the SVG export was designed nobody was probably thinking about 40,000 items and it worked well with this implementation. I am pretty sure that you will run into nasty performance issues if you use some other Paper.js features with your complex items. But that doesn't mean that the SVGExport's performance should not be improved.

[mvanga]

@iconexperience Ah that makes sense. So basically the parent is recalculating its own attributes every single time a child requests them to compare against. It shouldn't be too hard to cache this and tradeoff some memory usage (which one typically has plenty of) for computation time; I'll take a stab at this and see how far I can get (likely over the next weekend).

[lehni]

@iconexperience @mvanga I recently noticed this myself... It's far from ideal, and should be addressed. The best / easiest solution would actually be to implement style cascading first, as outlined here, and then just use this transparently to set the SVG styles...

[renschler]

I'm not sure if this is related @lehni , but I'm finding that for svgs with many paths, the importSVG() function is very slow when running headless with paper-jsdom.

The import works relatively quickly with this file in the browser, but I'm running it on the server with a 2 min timeout and it never completes. Svgs with fewer paths work fine though.

The svg I'm using is attached below: Woolf.svg.zip

Update: Also... @mvanga when you originally filed this issue it seemed like it was an issue specific to svg export when running paperjs headless with jsdom?

I'm not sure I followed the discussion between you/@lehni/@iconexperience, but it doesn't seem like the root cause discussed would be isolated to jsdom? Or am I missing something?

My issue with SVG import only surfaces when running with jsdom, things seem to work in the browser, so I guess it could be a different problem. If you all think thats the case I will file a separate issue.

[lehni]

Yes, jsdom is probably slow, and there is little we can do about it. We should try updating to its latest version though and see if that changes anything. Last time I tried that, it broke a ton of things in paper.js...

If performance is a concern, you could try running paper.js inside Electron or a headless Puppeteer instance alongside Node.js, for SVG reading / writing.

[renschler]

Gotcha. I'm running inside an AWS Lambda function, so what I ended up doing was using https://github.com/adieuadieu/serverless-chrome and then navigating to a local webpage that had my code embedded as scripts. This took a little while to figure out, but it's soo much faster than jsdom.

I first tried using the newest version of jsdom but you are right it was like whack-a-mole with errors so after a few I gave up and switched to the headless chrome approach.

[renschler]

@iconexperience @lehni @mvanga I wanted to take a stab at improving the performance here (not the JSDOM stuff, but the stuff mentioned here https://github.com/paperjs/paper.js/issues/1353#issuecomment-312636210).

Anyways, I'm looking at the code and I'm not sure how it works. I've written plenty of javascript but there's a bunch of new in stuff here for me.

(excerpt from https://github.com/paperjs/paper.js/blob/develop/src/svg/SvgExport.js)

Base.each(SvgStyles, function(entry) {
            // Get a given style only if it differs from the value on the parent
            // (A layer or group which can have style values in SVG).
            var get = entry.get,
                type = entry.type,
                value = item[get]();
            if (entry.exportFilter
                    ? entry.exportFilter(item, value)
                    : !parent || !Base.equals(parent[get](), value)) {
                if (type === 'color' && value != null) {
                    // Support for css-style rgba() values is not in SVG 1.1, so
                    // separate the alpha value of colors with alpha into the
                    // separate fill- / stroke-opacity attribute:
                    var alpha = value.getAlpha();
                    if (alpha < 1)
                        attrs[entry.attribute + '-opacity'] = alpha;
                }
                if (type === 'style') {
                    style.push(entry.attribute + ': ' + value);
                } else {
                    attrs[entry.attribute] = value == null ? 'none'
                            : type === 'color' ? value.gradient
                                // true for noAlpha, see above
                                ? exportGradient(value, item)
                                : value.toCSS(true)
                            : type === 'array' ? value.join(',')
                            : type === 'lookup' ? entry.toSVG[value]
                            : value;
                }
            }

These are pretty basic questions, but I think it would help my understanding a lot.

What is Base? What does Base.each do? I see Base.js and SvgStyles.js, but I don't see where Base.each is defined. Also what is exportFilter and what do SvgStyles represent?

[sapics]

Hi, @renschler ! Thanks for looking into this.

Original Base comes from straps.js which is created by @lehni . You could find Base.each function in https://github.com/lehni/straps.js/blob/master/straps.js#L55

[renschler]

thanks @sapics, I'm not sure if there is a paperjs project design doc but that would be super useful for new people who want to contribute. Just explaining at a high level what files do what, and what wrappers (like Base) are used would be very helpful.

For this problem, my entire SVG only has one style, so all I had to do was cut out the || !Base.equals(parent[get](), value) check. SVGExports that used to hang for over an hour now complete instantly.

That said, I wasn't able to figure out where the second downcall is performed? ( https://github.com/paperjs/paper.js/issues/1353#issuecomment-312630878)

my understanding was that parent[get]() would just return the parents value of the specified style, but does that trigger a node traversal of any sort?

[kgoedecke]

I'm having issues with this as well.

@renschler is headless chrome still your recommended way of doing this?

also I was wondering if there are any updates on JSDom 16 support? @lehni

[thexperiments]

Hi any progress on this? I'm not using jsdom but I would guess my problem has the same root cause.

I'm generating a halftone pattern for laser engraving and exporting takes 1-2min for a 5000 circle pattern. I could live with these 1-2 minutes (In Chrome on an Intel i7 8700k) but most patterns when using this would be much larger.

Any Ideas for a workaround to speed this up?

[dtbaker]

If you've got an example of running this in AWS Lambda with headless chrome @renschler that would be greatly appreciated. Hitting slow issues and jsdom whack-a-mole currently.