Add spline animation proposal.

[Pixneb]

Description of Proposal

Add spline animation to USD.

Contributing

• [X] I agree to and accept the Supplemental Terms.

[tallytalwar]

ready to land/review/iterate. (Merging soon via pixar-oss).

[JamesPedFoundry]

Hi,

I may have misunderstood some section, or missed some potential behaviour we could re-use. However, I wonder whether the scope of this should be slightly broader to make sure we don't duplicate our efforts when it comes to curve data in general?

For instance, shaders tend to have ramp or curve properties, which are a bit of a "wild-west" of attributes at the moment which this looks to be 80% of the way to defining. This proposal seems to handle animated values and attributes well, but seems to miss the potential for shared evaluation with a generic curve property value which isn't necessarily time restricted?

I don't envisage it being a paricularly different, other than removing time from being the key point of reference for knots. It would also require having some way to query the attribute system for the value of the curve at a specific float value rather than time based specifically.

However, it does also bring forth another question of whether per-knot interpolation is/should also be supported? I understand it's not widely used in animated values, but again for the case of shaders we have seen it being requested and used for shader curve definitions.

edit: I see it was mentioned in: https://github.com/PixarAnimationStudios/OpenUSD-proposals/blob/2bad4b86aa02185b5a54affed2e31c9f7792396e/proposals/spline-animation/README.md#time-orientation However, it's likely still worth pointing out the above use case which makes sharing shading data between DCC's difficult currently with no Universal way of describing a curve.

I'd still generally recommend time isn't the defined API, the holding our noses approach is fine if it was defined like that ahead of time, but given this is fresh and we know of the future use cases where "time" doesn't make sense, we could simply use a better word for describing the X position on the curve (position is something I have heard used, so a knot has a position and a value). Whether for now (as stated to get something in place sooner rather than later), the "position" is synonymous to "time", it at least keeps the door open to non-time based splines.

[Pixneb]

@JamesPedFoundry: can you give more detail on what "per-knot interpolation" means?

We do eventually want to support spline-valued attributes, but we think it will be survivable to leave them out of the initial release of USD Anim. Shader ramps are a great use case. Shaders are already being used in USD, and schemas have been put together to handle ramps, so it can be done. Those schemas are ad-hoc and inconsistent, and spline-valued attributes should improve them, but we want to get spline-driven animation working first.

As to time-oriented vs generic splines: the plan of record, so far, is to start with only a time-oriented API, because we want to avoid building in any generality that isn't yet used. We think swapping in a generic API with a time-oriented wrapper will be a reasonable option for the future, and we also think there would be no differences whatsoever in the implementation, only in the wording of the API. However, another possibility is to introduce only the generic API, and have even animation refer to "x" instead of "time". So far we're betting that would be too confusing, but we'd love to hear more feedback on the topic.

[JamesPedFoundry]

Re: @Pixneb.

I understand the position here, I would still prefer a more stable API over time, rather than having to refactor later when the API inevitably changes. However, I see the point where initial users of this may find the abstract names a little more confusing. I prefer the idea of starting with the time based API wrapper of an underlying generic API, such that we don't need to rewrite import/export logic further down the line. I imagine that would be more work, but might be the best of both worlds, with a simple interface for animated splines, but compatibility for future generic spline work. This is all just one opinion though, and certainly won't block your excellent detailed plan described here.

On per-knot interpolation, of course I can explain more! Essentially it boils down to being able to define the interpolation of each segment between with different interpolation methods. This may also be useful in animation, but is particularly useful in shading where you might want a gradual Bezier curve, followed by a linear interpolation to provide sharp line to signify sharp edges on the meshes (as one example).

It is likely much further down the requirements list and is likely something that could be added later if requested enough, and I understand it isn't supported by all renderers. An image below of a single curve that uses both linear and Bezier interpolation.

(Its a shame there isn't thread capability on GitHub comments. Apologies that I should have made the comment in the source to enable this, lesson learnt for the future!).

Thank you again for making this proposal, it looks like it'll be a great addition to USD going forward, and a lot of points well thought out.

[Pixneb]

@JamesPedFoundry: yes, top-level (non-)threads are terrible! See https://github.com/github/roadmap/issues/552

Certainly we intend to support per-segment interpolation methods. I've been working for so long with splines that do this that I may have neglected to state it explicitly. When I do my next round of doc polish, I'll make sure that's covered. Thank you for the pretty picture, it warms my spliney heart.

We are hoping that, even if we should introduce a non-time-oriented spline class later, the API will not change. Having decided to use a time-oriented API for animated attribute values, we will stick with that forever. The hypothetical future change would be that the implementation would switch to a non-time-oriented one, and then the time-oriented version would become just a header-only set of aliases for the same classes, methods, and parameters, just with different names. Both APIs would then be available, both doing the same thing, but with the signpost of time-oriented names when used for time-oriented purposes.

[dgovil]

Just forwarding an internally sourced comment that I'd been slow to post here

Splines of Vectors: I'd like to see this, but I think allowing things like translation to be specified as 3 separate floats solves a big driver for those. Another use-case I see is driving color attributes in shaders. Would be ok to have limited tangent controls here. This would allow types like vector3f and float3 to be driven by a curve whose knots are of those types

[meshula]

@dgovil is the motivation behind a desire for splines of vectors a correlated interpolation of some sort? examples ~

• color attribute - Hue change minimal, or constant brightness, interpolation?
• kinematic interpolation according to the Frenet-Sennet formulae

If so, would such a need alternatively be met by something like OpenExec and a constraint system?

[dgovil]

@meshula

Essentially, the thought was that if you had a vector3(like transform, color) that you wouldn't have to serialize them as multiple curves, but instead could essentially have the knots defined in the same type (so a curve of float3)

Anyway just wanted to forward that feedback along since the question of Splines of vectors was posited in the proposal. I can see the value in it, but I can also understand that it likely complicates things too, and API can probably sufficiently paper over things.

[Pixneb]

@dgovil Yes, splines of vectors is a borderline case: there are some potential use cases, so we can't say a hard no, but it's also rarely used (at least at Pixar), adds complexity, and is hard for users to reason about and use. We are confident, having implemented this already at Pixar, that it would fit into USD if we decide in the future that it's needed. We're going to try a first version without it, and see what people come up with. Thanks for the use case - colors are an interesting one, both in terms of how they're conventionally expressed, and in terms of how the interpolation will look, since our eyes are so sensitive to tiny color differences.

[dgovil]

Thanks, Deneb! Yep that all sounds good. Thanks again for taking on this effort :-)

[meshula]

Hi all, the proposal merged for easy access from the proposals front page! Like all the proposals not yet landed as part of OpenUSD, it is still open for comments. Suggestions that cover the whole proposal may be put forward via a fork and a pull request. More likely, you may have questions or suggestions about individual points; for these please open an Issue and use a permalink to reference portions of the specification you wish to comment on.