Add 2D transforms

[ajstarks]

Is your feature request related to a problem? Please describe:

Typical 2D toolkits include transforms

Is it possible to construct a solution with the existing API?

Not supported now

Describe the solution you'd like to see:

Include the standard affine transforms (translate, shear, scale, rotate) at least in the canvas API

[andydotxyz]

Can you please add a little more information, if possible with a use-case? I know that some transforms will be useful, but "other toolkits do it" is not reason enough in itself ;).

Perhaps image rotation stands alone as "clearly needed" but for the others it is less clear.

[ajstarks]

Transforms will be useful in gamedev for object positioning, scaling, and rotation

See: https://www.alanzucconi.com/2016/02/10/tranfsormation-matrix/

Also for general graphics programming it's useful and efficient to define objects once, and then apply transformations to them. For other applications it's sometimes useful to render rotated text

[andydotxyz]

If we have the Move() and Resize() on every CanvasObject then do those requirements (except rotation) not get catered for?

[Solander]

Some 2D transformations would open up for both game applications and more graphical applications. I'm making a program for planning things on a stage and rotating things would absolutely help in that. Looks like oksvg has transformation support from the first glance but not sure what library is used for the rendering of pixelbased images.

[andydotxyz]

We support standard Go images, so you can use any image manipulation tools.

You could use canvas.NewImageFromImage if you want it to scale, or canvas.NewRasterFromImage if you want to display pixel for pixel.

[AmanitaVerna]

I agree that it would be useful for a variety of applications to be able to transform images (e.g. with rotation, or flipping them, or the like). For instance, if you have a 'corner' image, and an 'edge' image, and you want to rotate or flip them to create the other 6 border images for a widget. Or let's say you're making an app where you can drag icons around and rotate them to face in any direction, like the stage layout planning program @Solander mentioned. Or you want to be able to build a dungeon-designing app or something for a TTRPG, and you want to use SVGs for your art, and you want to be able to scale things, rotate them, etc. There's lots of use cases.

I can see how to implement it in internal.painter.PaintImage and scaleImage (and modifying the svg cache functions), though the question there would be how to expose the ability to transform images in a public API.

One possibility would be adding a Transform rasterx.Matrix2D field to Fyne's Image type. Since Matrix2D is what oksvg uses for transformations, and since Matrix2D provides helper methods to rotate, scale, skew, and translate, it would both make implementation simpler and would make things easier for end-users, I think. The main problem I see is that matrices are unintuitive, and what happens when you do multiple transformations would also be unintuitive.

But, assuming the existence of a Transform field on Image, PaintImage could then do one of two things depending on whether the image is an SVG or not:

1. for SVGs, after calling icon.SetTarget (which sets up a transform matrix), if the image's Transform isn't the Identity matrix (it should default to that), call icon.Transform = icon.Transform.Mult(img.Transform). Also svgCacheGet would need to check whether the transform has changed, the same way it checks if the width or height have changed, and svgCachePut would need to record the transform in the rasterInfo.
2. for non-SVGs, instead of calling draw.Interpolator.Scale when the image's Transform isn't the Identity matrix, scaleImage could call draw.Interpolator.Transform instead, translating from the image's Transform Matrix2D to the f64.Aff3 that draw.whatever.Transform expects (e.g. f64.Aff3{m.a, m.c, m.e, m.b, m.d, m.f} where m is the Matrix2D). Oh, and when the transform isn't the Identity matrix, scaleImage would have to not just return pixels like it currently does if scale is set to ImageScaleFastest or ImageScalePixels.

A user of Fyne would basically only need to do something like myImage.Transform = myImage.Transform.Rotate(math.Pi / 8.0) to rotate by pi/8, or myImage.Transform = myImage.Transform.Scale(2.0, 0.5) to scale by 2.0 in the x dimension and 0.5 in the y dimension, etc. (Users might have to read about affine transformation matrices to understand in what order multiple transformations are applied and stuff, and what would happen when applying them)

The main trouble with this approach is that matrices are not remotely intuitive. The helper functions help, but their documentation doesn't make it obvious what will happen if you apply multiple transformations to the same image. That could be an issue if someone isn't familiar with affine transformation matrices, though we could write documentation to address it (or expect people to google and learn on their own how to use affine transformation matrices, which I would prefer not recommending, personally, because there's a lot of confusing pages about them (including wikipedia's)).

If users have to rotate/scale/skew/transform things themselves, without any new support in Fyne, here are some possible ways they might try to do so:

1. for SVGs, duplicate most of internal.image.painter.PaintImage and some other functions, in order to implement the changes I listed above, and pass the duplicate PaintImage to a Raster image, maybe.
2. try to write their own code to use oksvg to render svgs with transforms, which will likely come out looking wrong if they aren't using the same math that fyne is (which is why the first option is 'duplicate a bunch of internal code').
3. for non-SVGs, create image/math/f64/Aff3 affine transformation matrices in order to apply skews and rotations by arbitrary angles (and so on) to said images (note that Aff3 has minimal documentation and no helper functions) by calling draw.nearestNeighbor.Transform (or draw.CatmullRom.Transform).
4. read each pixel of an image and write them out into a new image, in a different order, in order to flip them horizontally or vertically, or rotate them by multiples of 90 degrees only. Or try to do other operations such as skewing, scaling, or rotation by arbitrary angles, but without using matrices, those operations will be more complicated to code (though not impossible and I'm sure there's plenty of code floating around out there to do all of them).

[andydotxyz]

This is some really good thoughts, but as you say it would only apply to images - I think the original request was to apply on any CanvasObject. If you only want to transform images then you could use an image package and pass the result to Fyne.

[AmanitaVerna]

FWIW I don't see any good way to rotate SVGs outside of Fyne and then pass them to Fyne, but maybe I missed something.

I spent some time thinking about and looking into transforming CanvasObjects, and the first problem I see is the software renderer. Even if we're only considering rotations, I think we'd have to write our own code to rotate them, and it would be pretty complicated and inefficient. We'd probably want to do it after the call to draw.Draw so that we have an NRGBA image to work with no matter what was being drawn - that'd make things a little easier, at least. In order to support rotations by arbitrary angles in software, we'd basically have to write code that first rotated the bounds (as four vertices) around the center of whatever is being rotated (which wouldn't necessarily be the center of the thing that was just Drawn), and then stepped through every pixel inside the transformed bounds, also rotating the pixel coordinates around the center of whatever is being rotated, in order to get the coordinates at which to sample the image which is being rendered. That's a lot of multiplications per pixel being handled in software, and a fair amount of work to write all that code.

It'd be simpler in OpenGL, of course, since for anything that is rendered as a texture (so most things), we could just multiply the vertices by the transformation matrix and then render the texture as usual, I think. I'm guessing it wouldn't be difficult to apply rotations to lines and the like, but I don't really know GLSL so I'm not 100% sure they're doing what I think they're doing.

We'd also need to alter the code that checks to see if the mouse is over something, to make it account for things being rotated.

I think if CanvasObjects were given the ability to be rotated, it probably wouldn't be useful to add scaling or translating capabilities, since CanvasObjects can be moved and resized already. The only use for scaling I can think of would be mirroring/flipping, which might be worth having separate methods for.

Also adding anything to the public API for CanvasObject would break existing code, so maybe it would be better to instead create a Rotatable interface or something, and implement it on BaseWidget and baseObject (along with writing the code mentioned above)?

Also I just realized that adding rotation would probably break everything no matter what just because of how MinSize and Layout work? I'm not sure how that could be addressed.

It'd definitely be useful (and cool) to be able to have this capability, but this all sounds like a lot of work to me. And this is without even getting into the possibility of applying a projection matrix so as to get a fisheye lens effect on a UI, the way some videogames have done. (Though if everything was being transformed, you could also presumably just transform the mouse coordinates in order to determine what was being clicked/hovered/dragged)

[chippyash]

I've just hit the issue of not being able to rotate canvas items. Simple use case; place text around a circular display rotated to be at a tangent with the circumference. So if votes are needed then +1 to have simple transforms available for canvas objects.

[hippodribble]

+1 for transforms on canvas items. Not sure where the line is between the fyne Canvas and rasterizing tools like gg, but whenever try to use them, I get massive memory leaks. Describing shapes in vector in fyne.canvas is fairly simple and effective, and I don't seem to get leaks.

It is easy enough to write your own transforms and transform vector point locations for lines, etc, but text can't be rotated, and I think UIs are the less for it. Equally, a Path canvas item which could be closed and filled would really open things up in a number of fields. You can roll your own, but it is such a common need that perhaps Fyne canvas is the right place for it.

The amount of extra functionality would be considerable for compositing vector art, creating custom widgets or layouts, drawing maps, etc.

Of course, you learn to work within any system and adjust your expectations. Fyne is capable of so much that I won't be dropping it any time soon. Please keep up the awesome work!

[satyavvd]

+1 for rotation/transformations of at least basic 2d shapes, instead of writing our own.

[alexiusacademia]

Is this still not resolved? I am having same requirements on my desktop app. I am currently creating a custom widget to create a graph plot and I want to rotate a text to display or represent the Y-Axis of the chart.

[andydotxyz]

You could do it through a graphics package like we do in FyneDesk https://github.com/FyshOS/fynedesk.

https://github.com/FyshOS/fynedesk/blob/7bc2c75eda3aeff73b18450fe32a656bc4af27c4/internal/ui/widgetpanel.go#L142

[alexiusacademia]

I'm a little bit new to fyne and still getting the hang of creating custom widgets. So far I wrote this in drawing nodes

// Draw the nodes of the graph.
func (r *ratingCurveRenderer) drawNodes(size fyne.Size) {
    r.objects = nil // Clear any previous drawings.

    var marginLeft, marginRight, marginTop, marginBottom float32
    // Set default values
    marginLeft, marginRight, marginTop, marginBottom = 70, 20, 20, 40

    // drawWidth := r.widget.Size().Width - marginLeft - marginRight
    // drawHeight := r.widget.Size().Height - marginTop - marginBottom
    drawWidth := size.Width - marginLeft - marginRight
    drawHeight := size.Height - marginTop - marginBottom

    // Calculate the scaling for X
    nodes := r.widget.Nodes
    minX, maxX := nodes[0].X, nodes[len(nodes)-1].X
    width := maxX - minX
    scaleX := drawWidth / width

    // Calculate the scaling for Y
    minY, maxY := nodes[0].Y, nodes[len(nodes)-1].Y
    height := maxY - minY
    scaleY := drawHeight / height

    // Draw the nodes
    for _, node := range nodes {
        circle := canvas.NewCircle(theme.ForegroundColor())
        circle.StrokeWidth = 1
        circle.StrokeColor = color.RGBA{0, 0, 255, 255}
        circle.FillColor = color.RGBA{0, 0, 255, 255}
        var radius float32 = 2 // Radius of a circle representing a node.
        x := node.X*scaleX + marginLeft
        y := node.Y
        // Flip the Y to simulate true graph coordinate instead of screen coordinate
        y = height - y
        y *= scaleY
        y += marginTop
        // Position the dot
        circle.Move(fyne.NewPos(x-radius, y-radius))
        circle.Resize(fyne.NewSize(radius*2, radius*2))
        r.objects = append(r.objects, circle)
    }

    // Connect nodes with lines
    /// ... connecting each two nodes with a line

    // Draw borders
    // ... drawing of box borders ...

    // Write axes titles
    axisTitleX := canvas.NewText("WATER DEPTH (meter)", theme.ForegroundColor())
    axisTitleX.TextStyle.Bold = true
    axisTitleX.Alignment = fyne.TextAlignCenter
    axisTitleX.TextSize = 12
    axisTitleX.Move(fyne.NewPos(marginLeft+drawWidth/2, marginTop+drawHeight+marginBottom/2))
    r.objects = append(r.objects, axisTitleX)

    titleYAxis := []string{"D", "I", "S", "C", "H", "A", "R", "G", "E", "(cms)"}
    titleYAxisHeight := len(titleYAxis) * 12
    var offsetTop float32 = marginTop + drawHeight/2 - float32(titleYAxisHeight)/2
    for _, letter := range titleYAxis {
        txt := canvas.NewText(letter, theme.ForegroundColor())
        txt.TextStyle.Bold = true
        txt.Alignment = fyne.TextAlignCenter
        txt.TextSize = 12
        txt.Move(fyne.NewPos(marginLeft/3, float32(offsetTop)+txt.Size().Height))
        offsetTop += 12
        r.objects = append(r.objects, txt)
    }

    // Draw ticks
    drawHorizontalTicks(r, marginLeft, marginTop, drawWidth, drawHeight)
    drawVerticalTicks(r, marginLeft, marginTop, drawWidth, drawHeight)
}

So yes, I drew the vertical text one letter at a time. Thanks for the quick response Andy

[andydotxyz]

Looks great!

[alexiusacademia]

Hey Andy, I think I got it working using the image rotation by github.com/disintegration/imaging. Will work on the scaling though since it's an image now but still works!

Thanks!