Keavon
commented
2 years ago @0HyperCube I figured you may want to finish up the transform cage with the last piece of the puzzle since the code is fresh in your mind. If this doesn't interest you at present, feel free to unassign yourself.
Currently the transform cage always remains axis aligned with the viewport (viewport space). Instead, we want the cage to display the bounding box of the layer in layer space. This mean that a rotated layer would show a rotated bounding box and a skewed layer would show a skewed bounding box. This will also help users (and us developers!) better visualize transforms.
In more detail:
Now that pivots are implemented and pivot-based scaling is working correctly, the last major improvement that's required to finalize this feature's UX is to make the bounding box based on the layer's bounding box and rotation, not the viewspace.
If one layer is selected, the transform cage rectangle should be the AABB for the layer itself, rotated with the layer's rotation and displayed in documentspace not viewspace (meaning the transform cage box will stay aligned with the layer when you tilt the viewport). This means that setting the pivot to the four corners will actually make the dot appear in the four corners of the rectangle, which is not currently the case if you rotate the layer or tilt the viewport.
If multiple layers are selected, we instead want the documentspace AABB of the layers. Not the AABB of the layer AABBs, but the shapes themselves as they are drawn in documentspace. For example, if you select two circles, rotating their layer transforms causing the corners of their own local bounds to point in a different direction shouldn't affect the combined bounding box. Tilting the viewport should tilt both the rendered artwork as well as the combined bounding box, since it's in documentspace.
Rotating a combined bounding box should maintain the rotation until a different layer is selected. You should be able to Shift-click other layers to add to the selection or remove from the selection and maintain the bounding box's rotation that resulted from a rotation (by spinning a corner or using R), but not holding Shift and clicking on another layer should reset the rotation so next time you have multiple layers selected, they are again aligned with the axes of the document.
To visualize which way is up, we also want the square corners/edges of the transform cage to be replaced with triangles that point up towards the top of the bounding box. For reference:
Specifications for the triangles
The triangle has a white interior and a 1px blue inset stroke width. Since we can't directly render inset strokes, here are the dimensions of the triangle including the centerline from which it has to be rendered.
Since the triangles go through the horizontal and vertical edges of the bounding box, they should be aligned as shown here by the light gray crosslines. This also shows the centerline described above and some pixel-sized squares for counting units: