search

dfki-ric / pytransform3d

3D transformations for Python.
https://dfki-ric.github.io/pytransform3d/
Other
604 stars 63 forks source link

Run-time additions to temporal transform time series #283

Open roshambo919 opened 3 months ago

roshambo919 commented 3 months ago

Thanks for this excellent work. My question is regarding support for handling incremental additions to the transformations within a temporal transform manager (TTM). From what I can tell in the documentation and examples, TTMs take as input an existing time series of transforms. However if we want to use a TTM at run-time, we won't know these ahead of time and will instead have to incrementally add to the TTM when new references come online at new times.

I can imagine a few ways to hack this together in the existing code, but I'm wondering if there's an existing approach that supports run-time additions to the transform history.

Thanks

AlexanderFabisch commented 3 months ago

Hi @roshambo919,

I never had this use case, but I am interested in exploring it. I guess you have to inherit from TimeVaryingTransform to create a new class that supports adding transformations dynamically (and maybe use this function to interpolate if needed). Everything else should work just like with the normal (Temporal)TransformManager.

If you are interested, we could work on a unit test or example to investigate the use case.

The temporal transform manager is a contribution of @kopytjuk. Maybe he has something to add.

kopytjuk commented 3 months ago

Hi guys, thanks for pulling me in, I always like to think & discuss new applications 👍

As @AlexanderFabisch suggested, I would implement a "ring-buffer"-like TimeVaryingTransform subclass, which holds a history of transformations and dropping old ones, when newer ones are incoming. So you keep the history short and look-ups quick. At least when we designed the interface of TimeVaryingTransform, we wanted to keep the freedom open for a developer like you, to use your own design decisions.

However, what I can deduct from the question of @roshambo919 is that you deal with a real time application, where a lookup back-in time is needed - so probably look-up performance will be important to you, as well - did you consider using ROS2 and its time-traveling features? The backend is compiled and the Python interface is a wrapper around it. The disadvantage is ofc. to have less control about the interpolation method, so you have to stick with the hard-implemented SLeRP.

Edit: ofc the "practically" bigger disadvantage is the ROS package as a heavy dependency which restricts your choice of operating systems and larger build times when doing CI/CD stuff ...

roshambo919 commented 3 months ago

@kopytjuk, you hit the nail right on the head.

I do deal with "real time" applications, but the majority of my use cases are in offline development and prototyping where real-time is not necessary. For most cases, a buffer is not needed and instead just using the most recent transform is appropriate, but there are a few cases where retrodiction/time-traveling could be useful.

We use ROS often for our real-time applications, but I strongly prefer not to use it for the offline prototyping/analysis with its package overhead and debugging challenges.

If I get some time, I can work on a PR and get your feedback.

AlexanderFabisch commented 3 months ago

If I get some time, I can work on a PR and get your feedback.

Yes, sure! My idea was to integrate it as an example or test case, but if it is useful and doesn't come with additional dependencies, we can also include it directly in the library.