Proposal: Allow custom Items

[georeith]

So I've had a series of hard problems recently working on a drawing program like render time pixel snapping. Fully style-able and editable text with wrapping, images with rounded corners.

I have been extending the base classes:

• For pixel snapping I extend the given Item and implement a _draw() method that rounds the bounds, then calls the regular _draw() for that Item then returns the bounds to their original state. Simple and only at render time, everything else can use the Item regularly and not care about rounding or any errors it may induce.
• Images with rounded corners are the same, I extend Raster but proxy the _draw method to an in-memory Group which contains a copy of the Raster and the Shape.Rectangle clipmask. The benefit here is that I have a single item in the scene graph, I don't have to teach all the rest of my code how to recognise the clipMask child of an Image.

However this required me to implement a fair few private methods. _getBounds(), _hitTestSelf(), _draw(), isEmpty(), setSize(). Which I know is taboo but far far outweighs the alternatives in my case for the ease and succinctness for which these Items work in a much larger system, the maintenance cost of keeping these in sync with the library would be much less than all the clunky code I'd need to litter the code with to handle them.

I think it would be a great idea for the library to expose an API for creating your own Items. A factory perhaps that requires an object with the minimum functions required for it to work effectively (mouseEvents, hitTests, getting bounds, drawing) and a guarantee that certain functions will remain stable and won't break in future releases.

[lehni]

I'm really weary of such a change... It'll close the door on a lot of API flexibility and possible refactoring in the future. I also don't know how this would translate to other backends...

[georeith]

@lehni I understand. Although larger libraries do refactor and users are expected to either upgrade or stay at a specific major version. It could take some of the load off yourself and the library

I've also used it to implement AreaText (using the SVG foreignObject method you came up with which works amazingly by the way) by extending Item with all the necessary methods and calling draw on an in-memory Raster just passing the Items context along.

Being able to tap into the draw function as a user is incredibly powerful. Even just exposing a way to enter that chain would be amazing allows you to do so much that is currently very hard in a very clean and DRY way.

Pixel snapping is the main one for me that would of have been a nightmare to implement across the entire application, but is so much more simple when proxying through draw and modifying the strokeMatrix so that it draws at a whole pixel (minus view transforms but that is what we wanted anyway).

[georeith]

Here is an example for snapping (It's not perfect yet but very close): Now the rest of the application can ignore it and not care that pixel snapping exists. Continue to use floating point math for everything else.

class Rectangle extends scope.Shape.Rectangle {

    constructor(...args) {
        super(...args);
        this.constructor = Rectangle;

        const baseDraw = this._draw;
        this._draw = (ctx, param, viewMatrix, strokeMatrix) => {
            const snappedStrokeMatrix = snapStrokeMatrix(this, viewMatrix, strokeMatrix || new scope.Matrix());
            baseDraw.call(this, ctx, param, viewMatrix, snappedStrokeMatrix);
        };
    }

}

function roundToNearest(number, n = 1) {
    return Math.round(number / n) * n;
}

function snapStrokeMatrix(item, viewMatrix, strokeMatrix, n = 1) {
    const snappedStrokeMatrix = strokeMatrix.clone();
    const { x, y } = item.strokeBounds;

    // We can't just round width and height here because the changing of left will alter
    // the right positional value and make it round differently. Such that if you position
    // another item at this item's non-rounded right the two values could round differently
    // and be drawn in different locations.
    const targetX = roundToNearest(x, n);
    const targetY = roundToNearest(y, n);
    const targetWidth = (roundToNearest(item.strokeBounds.right, n) - targetX) - (item.strokeWidth);
    const targetHeight = (roundToNearest(item.strokeBounds.bottom, n) - targetY) - (item.strokeWidth);

    // Store this as scaling is a getter on matrix decompose and we only need to it once.
    const currentScaling = item.matrix.scaling;
    const snapScaling = {
        x: targetWidth / item.bounds.width,
        y: targetHeight / item.bounds.height,
    };

    // Translate the strokeMatrix to remove the mantissa of it's x and y coordinates.
    // Dividing by the scaling factor is necessary as this needs to be in view pixel terms
    // and item's scale affects its actual size not the zoom factor.
    snappedStrokeMatrix.translate(
        (targetX - x) / (currentScaling.x * snapScaling.x),
        (targetY - y) / (currentScaling.y * snapScaling.y)
    );

    // Pre-scale strokeMatrix by the difference between the raw size and the rounded size
    const scaleMatrix = new scope.Matrix();
    // If strokeScaling is set to true during drawing the canvas context's transformation is reset
    // therefore we need to scale the matrix as if the item's position is 0, 0 otherwise we just use
    // the item's current bounds position translated into view coordinates
    const scaleOrigin = item.strokeScaling ?
        item.bounds.topLeft.subtract(item.position) : item.view.projectToView(item.bounds.topLeft);

    scaleMatrix.scale(
        snapScaling.x,
        snapScaling.y,
        scaleOrigin
    );

    return snappedStrokeMatrix.prepend(scaleMatrix);
}

[lehni]

Yeah I'm not saying I've decided yet what to do here. And there's nothing stopping you from already doing what you're doing. I'm not not allowing custom items currently :)

I would be very curious to see your implementation of AreaText, btw. I'm wondering if this is something that could find its way back into paper.js?

[lehni]

Another question: Since you're using class, wouldn't super work to call baseDraw?

[georeith]

@lehni I understand. I just wanted to share my experience of doing it and how much stuff we needed was solved by it.

AreaText I will ping you an email with it as there is a lot of custom functionality on ours too like focus to edit and I'm sure you'll want a more generalised one for the core, but I'll send it over and explain what's going on. It works really well for the most part almost exactly the same as the browsers own rendering with very minor differences in anti-aliasing.

There is still one problem I am struggling with and that is setting Rasters source even to a data-uri is an asynchronous action in a few browsers and its too late to do that a lot of the time as I can't halt the rest of the scene graph drawing while it updates.

Hence for stuff like initialisation and content changes it needs events so you can call view.draw() when it is ready.

Also about super you are right. Never thought of that, thanks!

[SebastianStehle]

@georeith Does your snapping solution also handle the problem with 1pt borders that do not look sharp?

[georeith]

@SebastianStehle It should do as it forces it to draw on a whole pixel instead of midway between two. Here is a 1px border in ours. Note I am using a mac retina display so not sure how it renders otherwise:

[SebastianStehle]

Yes, but the rounding is what causes the problems in my case. Lets say you have a rectangle (20, 20, 40, 40) (x, y, w, h)

then the left border is rendered from x:19.5 to x:20.5 ... so the following rectangle must be used: (19.5, 19.5, 39, 39) if borderWidth % 2 == 1

I dont understand how you handle this in your matrix calculation

[georeith]

@SebastianStehle Because I calculate from the strokeBounds.

I don't round width and height. I round left and right and then calculate the width between that. You can't do width and height because the same left and right point of two different elements would be different depending on where the other points are.

All I do is work out the new bounds I want and then the matrix I need to pass to draw to make that happen abusing the strokeMatrix argument to send it in.

The magic for dealing with strokes is here:

const targetWidth = (roundToNearest(item.strokeBounds.right, n) - targetX) - (item.strokeWidth);
const targetHeight = (roundToNearest(item.strokeBounds.bottom, n) - targetY) - (item.strokeWidth);

I work out the ideal left and right for the stroke to land on. Then subtract the strokeWidth from that which is now my target width. So target width may be a decimal but the overall strokeBounds when applied will be an integer.

With this though I also turn off strokeScaling. And manually step up the strokeWidth myself based on view.zoom in draw.

[SebastianStehle]

Oh yes, you are right. Nice :) ... I am going to test it. How have you integrated it? Is there an approach without creating subclasses?

Would also be nice to cache the results until the matrices change.

[georeith]

I don't know a way without subclassing (if you don't use ES2015 you can mimic this How does Babel.js compile a class declaration into ES2015) .

Currently I have a custom class that extends the Item I want to snap. Then I instantiate that class new MyCustomClass({ size: [100, 100], point: [50, 50] }) as I would the regular item and insert it into the layer or item.

Note there are a lot of other things you need to patch in your custom class though. Things like hitTest will now be wrong. Mouse events may be slightly off. It's not a quick win.

[georeith]

Regarding hitTest I haven't implemented yet but I believe the following will work.

• Extend hitTest.
• Work out snapped bounds.
• Work out current bounds.
• Find matrix that transforms snapped bounds into current bounds
• Transform the point by this matrix
• Call real hitTest with the new point

If you are doing what I'm doing with strokeWidth and scaling it you will also have to do it before the hitTest and then reset it after.

[SebastianStehle]

I see, I am thinking about making a factory for the items. It is probably easier to handle it there.