Numpy batched conversion operations

[machinaut]

Hi!

I'm using transforms3d in a bunch of higher-performance stuff than it might usually get used, so I've vectorized a bunch of operations (so instead of operating on individual angles, they operate on n-dimensional arrays of angles).

This works pretty well with all the interfaces except euler angles, where it's better to pass a (..., 3) n-dimensional array instead of having separate parameters.

Woud you be interested in a pull request that does this, and do you have preferences for what it'd look like?

Thanks, ~Alex

[matthew-brett]

Yes, absolutely I'd be interested. My main interest in maintaining backwards compatibility, and in keeping test coverage high. Would you consider doing a PR to give me an idea what you're thinking?

[machinaut]

here's a gist with some vectorized examples, can get this into a pull request in the next week or so

Notable difference is Euler methods have hard-coded rxyz convention, because I've hit so many bugs with folks mixing them up. PR would be generic.

https://gist.github.com/machinaut/dab261b78ac19641e91c6490fb9faa96

[matthew-brett]

Sorry to be slow to get back to you - the idea is good in general, but I would like to preserve the current function signatures for backwards compatibility. So, for example, I think euler2mat should continue to have the same signature:

def euler2mat(ai, aj, ak, axes='sxyz'):

and it should continue to return the correct result when these guys are scalars. The options for arrays are then to insist the user pass in their 3 by N array like this:

euler2mat(*arr, axes='rxyz')

or similar, or make the second and third arguments optional with defaults = None, and allow the first argument to be a 3 by N or N by 3 etc array.

[basil-huber]

👍

[vik748]

These vectorized operations would be very useful. Has this been incorporated in the repo?

[matthew-brett]

Sorry - no - I don't think anyone has had time to step up and do the work of rewriting the tests and code. It wouldn't be very hard, it's just that no-one has done it yet ...

[vik748]

I can work on it as and when I get time. I have vectorized quat2mat as a template in my fork at https://github.com/vik748/transforms3d/commit/45f1 . Can you take a look and let me know what you think? For 100,000 conversions the vectorized version takes 125ms vs 15secs a 100x improvement.

[matthew-brett]

Yes, broadly, something that would be fine. The key things are:

• The API has to stay compatible
• There have to be tests

It would also be good to avoid code duplication as far as possible. But, with those requirements met - yes - that would be great.

[vik748]

I think we need to make a decision on the style choice here when it comes to code duplication. The only way I can think of avoiding duplication is by converting a scalar or single input to the array, operating on the array and then reducing the dimension on the result. However, this would increase computation time on the single / scalar inputs. Here are example timings for quat2mat() on single quaternions:

current function - 3.45 µs ± 49.7 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)
function from commit 4571 (with duplication) - 3.55 µs ± 70.3 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)
function without duplication (convert to array, operate and convert back) - 70.1 µs ± 1.67 µs per loop (mean ± std. dev. of 7 runs, 10000 loops each)

So what is the preference, fork the function depending on input or convert scalars to arrays?

[matthew-brett]

I think it's OK to convert the scalars to arrays. The problems will only arise for people doing thousands of calls to the function, and then they can resolve the speed issues by upgrading to the array interface.

Before you go any further though - do you know about the Scipy functions, recently introduced, such as https://docs.scipy.org/doc/scipy/reference/generated/scipy.spatial.transform.Rotation.html ?

[vik748]

In some applications you have to perform individual calls, eg. if your data isn't available in batch for example. Say an IMU streaming data at a few 100 Hz, would be hurt by slow scalar operations.

I wasn't aware of the scipy rotation functions, when I had searched for a reliable package last year, this is the only one I found to do everything I needed. Thanks for pointing it out. Looks like it already supports vectorized operations and for mat2quat it seems to be faster than what I could do. So are you thinking about deprecating this package in favor of scipy? Have you tested the scipy package to see if it equivalent? Rotation related functions seem to have a lot of subtleties and I know from experience that Matlab functions can't do everything this package does.

[matthew-brett]

To be honest, I haven't looked into the Scipy functions in detail.

For my own uses, sometimes I need packages that are very small, all Python, and that have code that is pretty easy to read and modify - so my prediction is that when I've checked over Scipy, I'll end up putting a big pointer to the Scipy functions, and keep this one running for anyone who wants something small and in pure Python.

Are you intending to check the the Scipy functions through? That would be huge help, if you could report back.

[whophil]

I'm interested in this 2023, sorry for the thread necro.

The vectorized scipy stuff is nice, but it requires all of scipy to get it. Scipy also only supports rotations. Can anybody here recommend a lightweight alternative to this project which supports vectorized rotations/translations, if one exists?