Optimize Affine2D transforms

[QuLogic]

To address @timhoffm's other comment ("Does it still make sense to keep Affine2D._mtx as a numpy array if we only do element-wise operations?"): I'm quite convinced the whole transform stack would be faster if the transformation matrix was not a numpy array (because 3x3 is a size where the numpy's overhead is generally bigger than the gains from vectorization), but changing everything at once (even better would be to move things to C, but using plain C structs (or equivalently C++ objects) to store the coefficients) would be quite a big PR. So I'm doing the easy parts first :-)

Originally posted by @anntzer in https://github.com/matplotlib/matplotlib/pull/22119

[QuLogic]

I also noted that there exist several packages that do affine transforms already. I have not attempted any conversions, and they may not be any better, but might be useful as inspiration.

There are some pure Python packages:

• affine - Matrices for affine transformations on the plane
• matrix-47 - General matrix math

Some compiled options:

• affine-transform - Parallel affine transforms using C++17 and OpenMP
• Pyrr - 3D mathematical functions using NumPy (might be similar to ours though)
• cython-blis - Fast matrix-multiplication as a self-contained Python library (not affine specific)
• eigency - Cython interface to Eigen library

And some non-Python things that would need wrappers:

• cglm - OpenGL Mathematics (glm) for C (not affine specific, but does 2D and 3D math)
• math - Vector and matrix math in C++17
• Agg's builtin trans_affine struct

[QuLogic]

Since pybind11 has native support for Eigen, I've put together a branch https://github.com/QuLogic/matplotlib/tree/eigen-transform that attempts to use it. There are a few minor issues that need to be fixed, but it passes most tests now.

However, I've noticed that there is a cast to a NumPy array in the path extension here: https://github.com/matplotlib/matplotlib/blob/8f296dbde92cdcca7de69b1ceaa0f044c6529c5f/src/py_converters.cpp#L320-L323 which means we miss out on avoiding NumPy right now. This will be made easier once #27087 is complete.

I've not run any benchmarks yet, but at least this points out the pain points with doing this change (and we got #27511 out of it.)

[QuLogic]

Also, taking a quick look at Agg's trans_affine, it looks to be fairly similar to Eigen. It may be just as easy to implement that way, and not require the addition of Eigen.