Extend transformation functions

[ahausladen]

This patch adds functionality to the transformation functions, that allow to write the transformed image to a target-image instead of doing an inplace transformation. This reduced the the need to copy "array of TColor32" around or creating helper images, copying the whole source image into them and then transforming them.

Changes:

• Added: Transform functions can transform to a target TImage32 instead of modifying the source.
• Added: TImage32.Create(array, width, height) and TImage32.AssignPixelArray that allow to set an already existing "array of TColor32" as the fPixels array. (The TImage32 becomes the owner of the array)
• Added: TImage32.AssignSettings that copies the settings without changing the fPixels/fWidth/fHeight values.
• Added: TImage32.ResizeTo/ScaleTo that have an additional targetImg: TImage32 parameter
• Fixed: useBoxDownsampler detection. An image that is flipped by a matrix was only scaled down.
• Fixed: SVG TImageElement ignored the transformation matrix

[AngusJohnson]

Andreas, if you're wondering why this PR has been held up until now, your CanUseBoxDownsampler function caused me to ponder my own matrix decomposition functions (MatrixExtractXXXX), and over the last few days I have come to the conclusion that they aren't satisfactory and am still pondering how best to fix them.

I can't remember exactly where I derived my matrix decomposition code from, but it was likely from here or here. But over the last couple of days I've come to the conclusion that it's not possible (see here) to reliably decompose all 3x3 affine matrices (though it's certainly possible when the matrices contain only a single rotation and a single scale op). Anyhow, I'm hoping to upload some improved MatrixExtractXXXX functions soonish.

And WRT your CanUseBoxDownsampler function ...

https://github.com/AngusJohnson/Image32/blob/fbd54c8b13adc45a7c074e91916155d9bfaeb3b0/source/Img32.Transform.pas#L462-L465 unfortunatley the assumption that X scaling changes mat[0,1] and Y scaling changes mat[1,0] only holds true when scaling is performed after rotation. When scaling is performed before rotation then the converse is true. (EDIT: see my comment below.) However, ISTM that neither equation (mat[0,1]/sx = 0) or (mat[1,0]/sy = 0) should be influenced by sx or sy as only the numerators are meaningful here (and I suggest the other divisions would probably best be passed to the ValueAlmostOne function).

[tomwiel]

Hi Angus, it's not clear from your post how far your solution is. If you still need this (angle, sx, sy), I could provide a function in 1-2 days.

[AngusJohnson]

Hi Angus, it's not clear from your post how far your solution is. If you still need this (angle, sx, sy), I could provide a function in 1-2 days.

Thanks Tom. Yes, please .

procedure MatrixExtractRotation(const mat: TMatrixD; out angle: double);
begin
  angle := ArcTan2(mat[0,1], mat[0,0]);
end;

And the following is also more efficient ...

procedure MatrixExtractScale(const mat: TMatrixD; out sx, sy: double);
begin
  // https://stackoverflow.com/a/32125700/359538
  sx := Sqrt(Sqr(mat[0,0]) + Sqr(mat[0,1]));
  //sy := Sqrt(Sqr(mat[1,0]) + Sqr(mat[1,1]));
  sy := (mat[0,0] * mat[1,1] - mat[1,0] * mat[0,1]) / sx;
end;

And of course, these functions will only be accurate when there is no skew (which will need documenting both in the code and in the html documentation). Anyhow, I'd value your suggestions / improvements 😁.

[AngusJohnson]

Using the current code ...

  m := IdentityMatrix;
  MatrixRotate(m, NullPointD, angle90);
  MatrixScale(m, 2, 1);
  MatrixExtractScale(m, x, y);

x =1 and y = 2

  m := IdentityMatrix;
  MatrixScale(m, 2, 1);
  MatrixRotate(m, NullPointD, angle90);
  MatrixExtractScale(m, x, y);

x =2 and y = 1

And reversing x and y makes sense following rotation, but ISTM that x should equal 2 in the first code block.

EDIT: OK, I think I've finally gotten my head around this. Rotating 90deg before scaling width x 2 causes the original image to be stretched vertically, hence y = 2 is correct as per the first code block. Stretching the image width x 2 and then rotating 90deg, causes the original image to be stretched horizontally, hence x = 2 is correct as per the second code block. I'm not sure if I'd figured all this out before and had forgotten it, or if I never fully understood it. Probably the latter 😱.

[tomwiel]

Disclaimer: Although I have some expertise in this area, it's not much deeper than this what I'm writing here. I think the challenge is that the matrix can contain any affine transformation, which can result from any userdefined operation sequence (userdefined matrix-product (A B C ...)).

This has several consequences: 1) There is no way around some decomposition method, because shortcuts would fail (simple extractions would fail on non-orthogonal matrix). But this small slowdown is neglectable, because such functions are not called per pixel.

2) Even correct decomposition (any type) is only of limited use for end-users: Decomposition generates an alternative matrix-product (but producing the same visual result). If this alternative matrix-product is different from the userdefined matrix-product, then decomposition will often return strange-looking (although correct) parameters. Example: If a rectangle is first rotated, then nonuniformely scaled, a skewed rectangle is displayed (is correct). The decomposition can extract (fully regain) that userdefined scale only if the decomposition is (coincidently) defined like the userdefined matrix-product. But then a different problem occurs: The userdefined scale is fully obtained, but the extracted skew is zero (which is correct only in this context). Now the inexperienced user could think, the tested matrix contains only scale and rotation, but no distortion. Another limitation: It's not possible to get five different parameters (angle, sx, sy, skx, sky), but only four (due to 2x2 matrix). Extraction methods (if public at all) need documentation about the decomposition matrix product, to prevent surprises.

3) Due to mentioned limitations, we should use extraction only sparingly; for getting scale; for query if matrix has rotation or skew. Whenever possible we should directly use the current matrix for drawing (without extracting parameters).

I've added some extract functions (not fully tested yet, so don't publish). Will test them later today or tomorrow. extract.txt

Should be OK now. if the reverse product (M = Z2 Q) is wanted, then 3 lines need to be changed: scale.x := a r + b t; scale.y := c s + d u; skew.x := c r + d * t;

[tomwiel]

Using the current code ... x =1 and y = 2

This looks indeed confusing, probably a bug. Transformations also look confusing due to many different conventions (SVG, GDI, OpenGL, ...)

[AngusJohnson]

Using the current code ... x =1 and y = 2

This looks indeed confusing, probably a bug.

I thought so too until I looked at the issue more closely. What I did was start with an image with an easily discernable top and bottom (irrespective of subsequent rotation). I rotated this image 90degrees and then scaled it - width x2. The end result visually was exactly the same as when I scaled the image - height x2 - before rotating 90degrees.

[tomwiel]

I see, and this is because the img32 transform convention is reversed to SVG sequence: Your code (matrixRotate(); matrixScale()) is equivalent with: svgScale(); svgRotate(); SVG adds systems from left (parent) to right (child) to produce the same current matrix, Your example corresponds to my extraction method (M = Z2 Q). So Z2 would deliver the userdefined scalefactor (sy = 1), Extraction method (M = Q Z) delivers the visual (sy = 2), which is Ok as well.

svgRotate() with current code would be:

procedure matrix_addLocalRotate( var matrix: TMatrixD; angle: double);
var mat: TMatrixD;
begin
  mat := identityMatrix;
  MatrixRotate( mat, NullPointD, angle);
  matrix := matrixMultiply( matrix, mat);
end;

[AngusJohnson]

I see, and this is because the img32 transform convention is reversed to SVG sequence: Your code (matrixRotate(); matrixScale()) is equivalent with: svgScale(); svgRotate(); SVG adds systems from left (parent) to right (child) to produce the same current matrix,

That wasn't intentional, and is obviously confusing, and needs fixing.