Constraint approaches, the book.

Part1: Reminder of what we're doing with our UI

We're creating a CAD UI that treats code as the source of truth. In any software where a user is producing an artifact (that is video editing, ux-design or CAD software as apposed to project-management, or email UI as an example) as the user interacts with the software they are building up more and more data. Normally this data is stored in some proprietary and obscure datastructure. We're choosing to instead use code itself as our data store for the following three reasons:

Works with our code-first philosophy
It's as "respectful" as we can be to our users by giving them a 1:1 mental model of how the software works
Is much better at capturing intent of the user
Enables engineers to leverage CI style pipelines, another goal of ours, because what they get out of their work is code that can execute in the UI or a pipeline.

The way this works is that the code executes and produces artifacts for the 3d scene. When users click around and modify things in the 3d scene, they are not modifying the scene directly, they are modifying the code that executes again to produce an updated scene. There is always a strong link between the code and the 3d scene, achieved by attaching source-range metadata to the 3d artifacts so that when you hover over a line of code, you can see the corresponding 3d artifact and vice versa.

Part2: Why 2d sketch and extrude is so important

We're aiming for 2d sketch and extrude workflow, simply because:

It's industry standard, and very intuitive to current MEs.
CAD design is centred around manufactured products, and this means that designs are very exact, everything is dimension. A 2d to 3d workflow suits this very well for the majority of functional parts. I.e. we get a high engineering-time ROI out of this relatively simple workflow, we can extend with more exotic features after the fact.

Part3: How this would work with 2d solver approach.

The current norm in CAD software for sketches is to roughly draw your sketch, this gives it some rough initial values and rough initial shape, and from there the user adds a series of constraints. Constraints are relationships between segments of the sketch that can be expressed mathematically. A 2d sketch solver can then tell the user if they have enough constraints to uniquely determine the shape of the sketch, and solve the sketch if so. The inner workings of the solver are never exposed to the user, Even though CAD-users are very competent in reasoning about geometry, they are completely abstracted away from this. It's also worth noting that the initial values of the segments in the sketch are completely disposed of by the time the sketch is fully constrained, They serve only as a starting point, this has to be considered when thinking about the API, and code-gen aspect for our UI. Let's run some thought experiments, let's say we're sketching the following parallelogram

Imagine the user clicks around in a loop to create the shape roughly, they came close to the dimensions they need, but not quite. A hypothetical API for how this is expressed as code is to add a number of segments with rough values, they are defined as polar vecs here, but the exact way of defining isn't important. I've also used a piping syntax, but they could just as easily be in an array, again not really important when thinking through this hypothetical API.

After the sketch has been roughly defined it's time to add constraints, with this 2d solver approach these constraints will not be inherent to the segments themselves, instead they will be attached later as overrides for the original segments, from a code-gen perspective though it's important for us to remove redundant values when the constraints are added as having stale and therefore possible confusing code is undesirable. Let's start with defining some "equal length" constraints

Here the user would have selected the 'A' and 'C' segments and clicked the 'equal' constraint, and so the code now conveys that, i.e. segments 'A' and 'C' are equal, but we also want to remove the length values from when those segments were originally defined, so now they have become polarVec({ angle: 30.02 }, 'A', %) and polarVec({ angle: 209.02 }, 'C', %). We still need the length of the lines, but since it's not clear which length line should take preference instead a nominalLength has instead been added to the constraint itself, this is a little messy as once we add some other length constraint we'll probably want to remove this, making it another awkward temp value. Let's add the next equal length constraint for 'B' and 'D' segments.

Very similar to the last, we'll move one with a different type of constraint, equal length.

In the above, we've added two constraints to speed things along. They both are effectively length constraints (one is yLength just so the constraint is in values from the original dimensions). When adding these constraints the modify-ast-helpers needs to be aware of the temporary values that need to be removed. In this case the "nominalLength" values from the first two constraints.

Let's add the final constraints

Here we've added angle constraints for both 'A' and 'B'. Segments 'C' and 'D' don't need to be constrained by angle as after all of the other equal-length constraints have been applied they will be the correct angles. Notice that the temp angles that were used in the original line declarations have now been removed because they are not needed anymore. This means that those lines are nothing more than line labels, ABC & D, with the constraints using those labels to actually define the shape it creates, I've also changed the function call name to just vec as polarVec has no meaning anymore.

I do not like this API. It's hard to read due the disconnect between where the segments are defined and then how they are constrained. Some of the tooling I've added to connect code with scene-artifacts doesn't work well with this API, for example:

If one hovers over a segment in the scene what lines of code should it highlight? both the lines where the label is given as well as any constraints that use the label the line is given?
If the user puts their cursor in the editor within vec('A', %), then we should highlight that segment in the 3d scene, but should also highlight other parts in the code, i.e. the constraints that mention 'A'? it's not clear to me if that makes things better or more confusing.

One other concern I need to ask is "is that sketch actually fully constrained yet", it's a little hard to answer, let me explain: When adding sketch constraints in normal CAD, some nuances are left out, as dimensions are always absolute, i.e. you wouldn't give something a length of -5 (well yes you can, but that's to flip it's direct, but after you've done so, when you come back to edit again it will be 5 again), for angles this takes the effect of 30° meaning the same as 210° (30 + 180), or that line 30 degrees from the y-axis, this could be CW or CCW from the axis. And if we keep the same rubric then really the sketch we've constrained might have as many as 8 solutions

I've layered so many sketches on top of each other it's hard to see what's going on but you get the point. I imagine that CAD packages must keep information about the directions of these dimensions that are never exposed to the user. In our case since the code will need to produce the same 3d artifact, we'll need to be a bit more explicit about these things. This might mean adding an extra parameter on the constraints to specify direction or using the sign of the value to indicate the direction.

My concern with adding extra information to denote direction is it smells, it feels like an awkward level of abstraction, the user need not concern themselves with how the lines get constrained/solved, because the solver takes care of it, and yet the values that end up the code don't match 100% with what they entered. (i.e. they specify 5, but appears as -5 in the code or similar).

One other very tentative concern I have with this approach is that I fear it's not compatible with branching logic. Branching logic is going be a very powerful feature for our users as it will allow them to make their models very robust to even dramatic changes in input parameters because they determine how the model will adjust with those changing params. if statements will allow them to add breakpoints as such to adjust the model. This will likely take the form of removing or adding sketch segments depending, but because the lines and their sketches are defined separately and because constraints involve multiple segments, and the final result needs to be a sketch that's not under or over-constrained and doesn't reference segments that don't exist anymore. Therefore branching logic sounds tricky with a 2d-solver approach to say the least.

Part4: How this would work with constraints embedded in the line definitions.

What has already been implemented (but not finished) is a constraint system that is much more code-centric. Instead of constraints defining how segments relate to each other, each segment is defined fully in the same line (line of code) in which it's created, referencing previous segments in order to tie certain values together i.e. line({length: somePreviousline}) to make things equal length or line({xVal: somePreviousLine + 10}) to make the segment 10 units apart on the x-axis. Segments can also be defined using a variety of parameter types which makes it easy define the segments with the dimensions that seem most intuitive, for example:

line(...) takes xy coordinates that are relative from the last line
lineTo(...) takes xy coordinates absolute to the sketch plane
angledLine(...) takes polar coords angle, length
angledLineToY(...) takes a combination, it has a particular angle, and will be as long as it needs to be to get to a specific y value
so and so forth

This implementation also uses a variation of magic-number concept to determine if segments are constrained. In programming magic-numbers are hard-coded numbers that have no explanation from the surrounding code and therefore there's a high risk that the purpose of the number will be forgotten with time, making it hard to maintain. For example let's say you find the following line while trying to fix a bug const fluxCapacitance = COOL_CONSTANT + 156.43, what is 156.43? it's very specific so it probably has a good reason to be there but a programmer has no idea from reading the code. I can be improved by simply naming it.

const radianceFactor = 156.43
const fluxCapacitance = COOL_CONSTANT + radianceFactor

Now that constant has heaps more meaning. Likewise in the UI, when first adding the sketch segments, initially they have a lot of hard-coded numbers, or "Literals" to be specific of how the language AST refers to them. The UI treats the literals as unconstrained values, and will happily modify them further, either when the user clicks-&-drags the head of a segment, or when adding "constraints". However when adding constraints the "literals" are transformed into CallExpressions, Identifiers or Binary expressions (that is function calls, referencing a variable, or math equations 1 + 4). Once this is done that value is considered constrained, but also now has more meaning, the approach can be thought of as "You should reduce your magic numbers, and constrain your sketch", as a way to both embed meaning into and properly define your sketch.

With this style of constraint, there is also a mental model and best practices for applying constraints.

Mental model is that:

Each segment has a head and a tail
Each segment starts from the previous segment
Each segment has access to the previous segments and can reference them (i.e. backward references), however, forward references become problematic and should be avoided.

Therefore

The best practice for defining constraints is to either:

Fully constrain each segment in the same order as they appear in the sketch referencing segments pior-to/already-constrained.
Alternatively constraint the angle of each segment in the order they appear in the sketch, then define the length of each segment in the order they appear in the sketch, again referencing segments pior-to/already-constrained.

Also:

Don't try and define a segment's length as well as it's endpoint

Not strict, but following these rules will ensure things flow well.

Let's walk through the same parallelogram example but with this alternate constraints approach. Since this has already been implemented we can use screenshots from the UI. Starting off with the initial draft shape, lots of literals in the sketch at this point.

Let's start by constraining the top horizontal segment to be horizontal, The segment is selected by clicking on it, but what this actually does is put the cursor on the relevant line-of-code, that way the ast-modification-helpers know what part of the code needs to be modified. We can constrain it to be horizontal by clicking the 'horz' button after the segment has been selected.

Notice now that line 3 in the editor has transformed to an xLine function call, this only takes a single x-value as it's been constrained on the yAxis, further the segment in the scene has turned red, this indicates that the line is partially constrained, its length can changed but it must stay flat on the xAxis.

Now let's specify the angle of the first segment, here we're selecting that segment and the y-axis, in order to specify the angle in the same way it is annotated in the original drawing. We're also given UI to prompt us of the angle, originally it told us from the rough sketch that the angle of this line was 26°, I've changed it to 30, notice that it's also indicating that the angle is negative, that's because from the y-axis to the line is in a CW direction, but the user doesn't have to worry about that, it's just there as an indication. We're also going to create a variable for this angle.

After applying the constraint, that segment now has also turned red to indicate it's partially constrained, and in the editor, line 3 has been transformed from a line to an angledLine function call, as this better accommodates defining the angle of the line, notice the angle is defined as _90 - myAng that's because the angles are defined form the xAxis, and so by selecting the yAxis we've added 90° to align to the yAxis, than reducing by myAng/30 to get the angle of 60° from the xAxis.

Let's define the angle of the last segment (close doesn't count as it just connects the last to the first) by making it parallel to myAng, we can select those two sloped angles and hit the parallel button.

The result of this is that on line 6 of the editor, a tag has been given to the original angledLine so that it can be referenced. then on line 9, the line has been changed to angledLine similar to last time, but the angle is defined as segAng('seg01', %) + 180 that's because the interface is very explicit about these values, and the second sloped segment is actually 180° different from the first when taking the tail and head of the segment into consideration. The parallel constraint button will snap the line to the nearest 180° to make them parallel. The segment has also changed colour to indicate it's partially constrained.

Let us now turn our attention to defining the length of the segments to fully constrain them. Starting with the first sloped segment, we'll select it and the xAxis to define its height to 5, to match the drawing, we'll create a variable here too.

On line 4 of the editor, the angledLine has been transformed to angledLineToY as this better accommodates how we're defining the line, and on line 6 the to property has been assigned the height variable, also the segment has also now turned green to indicate that the segment is fully constrained, that is the function call uses all non-literal values.

Next we can set the length of the top segment, which can be done by selecting it and using the 'setLength' button. On line 5 of the editor the xLine did not need to change, but the literal value of 5.04 has been swapped out for our variable of the dimension width. The segment has also turned green to indicate that it's fully constrained.

Lastly let's set the length of the last segment by selecting the first sloped line and the second and hit the 'equalLength' button. On line 13 of the editor, the length of the line now references the first slopped segment, and the segment has also turned green to indicate it's fully constrained.

We're finished with this sketch, as a bonus, by putting our major dimensions into variables, it's now easy to modify the sketch in regards to those parametric values, for example:

Since I can actually do the demo we just went through, here's a clip of it from start to end.

https://user-images.githubusercontent.com/29681384/230824723-2327c720-f433-4caa-8915-87438c7052fc.mp4

It's worth reflecting on how readable the code is that's been generated, the first segment is

angledLineToY({
  angle: _90 - myAng,
  to: height,
  tag: 'seg01'
}, %)

This reads well as a segment that's defined by it's angle and it's height. Then:

xLine(length, %)

Reads as a horizontal segment that has the length of the variable used

angledLine([
  segAng('seg01', %) + 180,
  segLen('seg01', %)
], %)

This segment reads as a segment that has the same angle as a previous segment except it's going in the exact opposite direct, and it also has the same length as that previous segment.

close(%)

Closing off the loop.

It becomes even more readable when we take into account that we can hover over the segments to show us the code that we're concerned with.

The fact that the code is readable is no small thing. If we're trying to give our users a really clear mental model for how the software works, then it's very important that the code produced is intuitive.

What are some of the problems with this approach?

There are a handful, but they can be summarised by "It doesn't replicate the constraint workflow ME are used to exactly". I think this can largely be mitigated by communicating the mental model of the sketches and the best practice for how to use the constraints.

Can't do forward references. In most CAD-packages, you can add constraints in any order you like, but because this method works on backward references only you can get yourself stuck. Take the following situation where the second slope segment has been constrained with an angle first.

Here a user might select both of the slopped segments and expect to be able to apply a "parallel" constraint, but with the current setup that can't be done as the second segment needs to reference the first, a few comments on this situation:

The easy way out of this situation is to simply select the first segment and the yAxis and use a "setAngle" constraint to set it to the same angle, and now these are effectively constrained the same, obviously that doesn't help the users who is simply confused as to why the "parallel" constraint can't be set
We could definitely add some UI elements that indicate why this isn't possible and suggests what to do about it.
Lastly we can probably actually just solve this case, in that the second segment is using an angledLine function call, meaning we can simply grab the angle from it, using that value again in the segment we're trying to constrain. Because the second segment is using angledLine and we want to set another segment to the same angle, that makes this forward-referencing case relatively simple, and so it doesn't generalise to more complicated forward-referencing cases.

Let's look at a more complicated example, I'd describe this problem as "not being able to constrain a line more than once" but it also involves forward referencing. Let's say we're trying to sketch the following (some dimensions are missing here, but the important ones to demo the problem are there.)

Below I've constrained all of the segment angles, but none of the lengths

Now let's say I think it makes sense to constrain the far-right vertical segment to being 0.5 long. Great, that works.

After which I might want to select that same vertical line and set where it's head ends by selecting it and the xAxis.

But when I do so, non-of the constraint buttons at all are available 👆. That's because that vertical segment is fully constrained, it has a set length, where its tip end is instead controlled by the last segment that has a vertical component. See the clip below, it happens to be the slopped segment.

https://user-images.githubusercontent.com/29681384/230832639-a3191b94-f300-4e78-b447-9516a655b407.mp4

In order to get the result we're after we'll need to select that sloped segment and the xAxis and set it to 2-0.5 that is the total height we're after 2 minus the thickness of the arm thing 0.5

⬇️

I really don't see this as an insurmountable problem for ME's once they have the correct mental model, it's pretty easy to reason about. But it might be possible to have to enable forward referencing that help in a situation like this.

const myAng = 45
const armThick = 0.5
const part001 = startSketchAt([0, 0])
  |> xLine(2.61, %)
  |> tempAngledLine({angle: -myAng, tag: 'myLine'}, %)
  |> xLine(3, %)
  |> yLine({length: -armThick, tag: 'otherLine'}, %)
  |> constrainPreviousLine({refernceTag: 'myLine', toY: 2 - segEndY('otherLine', %)}, %)
  |> xLine(-8.08, %)
  |> close(%)
show(part001)

Very hypothetical, but we might be able to generate that code from clicks, including selecting the yLine segment after it's fully constrained and set where its endpoint is.

Part5: Am I straw-manning the 2d solver approach?

I sure hope not, an API or code friendly approach to 2d-solvers has been something I've been thinking about and had on the back burner for years (for the morbidly curious there's also this and this. And my move away from 2d-solver approach was born out of the fact that I never figured out an elegant and expressive API, and the conclusion that they are a good solution for the black-box nature of normal CAD GUI, but not a good solution for the more explicit paradigm that we're pursuing.

Another encouraging fact that makes me think that it's not just my failures to come up with an expressive API, is that since then CadQuery has released a 2d-constraint solver in their library and it's pretty much the same, and just as messy, segments added first with names, then a serious of constraints added after, and it's just as hard to read. The following example is from their docs and it's as simple as they could make it, only two segments, and yet the code is verbose and hard to follow. No shade here, I think they're simply bumping into the same problem.

Having said all of that, we have lots of smart people here, if you have a good idea for an expressive constraint API, let's hear it!

Part6: Summary

When it comes to the UI, I by far have the most context on where things are currently at and alternate approaches, so the aim here was mostly to try and share as much of that context as possible so we can have a productive conversation. On the issue of 2d-solver vs not constraints, it's already probably pretty clear that I don't think teaching users our sketching mental model and some best practices is too onerous and the benefit of getting much more readable and deterministic code makes it worth it.

Hopefully, I've given enough context for us to figure it out together.

Part7 Appendix: Is there going to be other contentious APIs going forward?

Our aim with the UI is to meet MEs where they're at, and right out the gate you're proposing something that departs somewhat from that.

I actually don't think there's risk of this happening again, sketches have a very unique place in CAD software, and the problem with solvers I'd mostly put down allowing arbitrary references (backwards and forwards). I don't think we'll have this problem again.

KittyCAD / modeling-app