ImageEngine / cortex

Libraries for visual effects software development
Other
531 stars 124 forks source link

additional Spline flexibility #1119

Closed ericmehl closed 3 years ago

ericmehl commented 3 years ago

I'm interested to know if there is any interest in expanding the range of splines that can be represented with IECore::Spline. Using the std::multimap built-in sorting works for linear and Catmull-Rom splines but Bezier and (I think) BSplines can still have a unique y for every x even if their control points' x-values are not strictly increasing.

Here's an example: https://www.desmos.com/calculator/mgpdl4waxm

I imagine there may be problems with changing the current implementation since it would likely require removing public access to the control points, adding checks to make sure newly added points don't create an invalid spline, downstream changes in Gaffer, etc.

It came up for me when implementing Gaffer::SplinePlug for a shader that can only accept Bezier splines. Converting a Catmull-Rom spline, for example, can potentially create valid Bezier splines with non-monotonic x-values. I can work around it by not using IECore::Spline for my converted Bezier splines but I thought I'd see if there is more widespread desire for changing or adding to the current spline representation.

johnhaddon commented 3 years ago

@danieldresser-ie might have other thoughts on this, but our main use for Spline is to provide values to OSL (and formerly RSL) shaders, and the OSL spec says this :

Results are undefined if the knots do not specifiy a monotonic (onlyincreasing or only decreasing) set of values.

Without a broader set of use cases for non-monotonic-x, I think I'm fairly reluctant to switch things up and add complexity. I wonder if you can elaborate more on your specific use case to see if there's a simpler or less-invasive approach we can recommend?

ericmehl commented 3 years ago

Ahh, I didn't connect that it was directly related to OSL, so it makes sense now to leave it as-is. If I put on my pedantic hat I would argue that the requirement by OSL for the x-points to be monotonic is too restrictive and should instead be imposed on the interpolated x-values. But that's opening up an even bigger can of worms.

Anyhow, my case is that VRay only supports Bezier splines in shaders, not Catmull-Rom, etc. (For its native shaders that is, OSL is supported also but I'd like to implement VRay-native shaders as thoroughly as possible). There are a couple of different curve shaders (taking a float / color input, outputs float / color, just like OSL splines).

The simpler one takes an array of 2d vectors where every four vectors define a Bezier segment. The Cortex spline types can all be converted to Bezier so I can support all of the current Gaffer::SplinePlug types with some math trickery. These mostly worked great but I found some edge cases in testing where the conversion didn't work and that's what led me to the sorting issue.

For example a Catmull-Rom spline from a SplinePlug with user-facing coordinates at 0.0, 0.0, 0.15, 0.6 and 1.0, 1.0 gives an IECore::Spline with the expected extra end points: 0.0, 0.0, 0.0, 0.0, 0.15, 0.6 and 1.0, 1.0, 1.0, 1.0

Converting this to a Bezier curve results in points at 0.0, 0.0,, 0.025, 0.01, -0.1667, 0.4333, 0.15, 0.6 (segment 1) 0.15, 0.6, 0.31667, 0.7667, 0.8583, 0.9333, 1.0, 1.0 (segment 2)

The -0.1667, 0.4333 coordinate is where the sorting throws it off even though it's a valid Bezier spline.

At first to test the conversion I was returning an IECore::Spline with bezier basis from my conversion function so I was able to confirm it was accurate by checking a lot of points between the two. With the point sorting breaking some cases I moved to just returning a vector of points that I plug into the VRay shader and verify through an actual rendered image, albeit with some expected numerical error.

Ideally I'd like to be able to have an interface for the full range of splines VRay can support. Since it uses a full set of four points (no shared points) per spline segment, it can represent corners, curved-in / linear-out and similar.

At that point I start to think about similarities to the animation curve editor and how it might make sense to generalize it into a plug or widget of some kind to handle Bezier splines with different point types and ranges greater than 0-1. I also think about how that would be a pretty significant undertaking, but might be worth it?

johnhaddon commented 3 years ago

With the point sorting breaking some cases I moved to just returning a vector of points that I plug into the VRay shader

Cool, glad you have it working. I can see that it isn't ideal, but I think this is the best approach for now.

At that point I start to think about similarities to the animation curve editor and how it might make sense to generalize it into a plug or widget of some kind to handle Bezier splines with different point types and ranges greater than 0-1. I also think about how that would be a pretty significant undertaking, but might be worth it?

That does sound plausible, and we certainly do need to put a good bit of work into Gaffer's animation editor. I'm afraid I'm a little short on time to be giving this much thought right now, but perhaps we can discuss at the point the animation editor finally rises to the top of the list?

ericmehl commented 3 years ago

Sounds good. When it does rise up, and if I have available time, I'd be happy to contribute if it's feasible to delegate.

I'll close this for now since my original question is taken care of.

Thanks John!