Backwards compatible additions to variants

[badeend]

Some enums are "open ended" in nature. Imagine a function that conditionally returns an enum containing the error code. For example:

(type $Errno1 (variant (case "acces") (case "badf") (case "busy")))

There is no way to add new error cases to that enum without breaking backwards compatibility, since the enum is used in a return position. Note that this problem does not occur when the enum is used as a parameter.

As it stands with the current Interface Types proposal; to work around this limitation we have to resort to returning meaningless integers (or any other type of artificial constant):

const Errno1_access: i32 = 1;
const Errno1_badf: i32 = 2;
const Errno1_busy: i32 = 3;

I suggest adding some kind of annotation to variant types that optionally marks a single case as the fallback that future additions resolve to. For example:

(type $Errno2 (variant (case "acces") (case "badf") (case "busy") (case "unknown") fallback "unknown"))
(type $Errno3 (variant (case "acces") (case "badf") (case "busy") (case "eagain") (case "unknown") fallback "unknown"))

Because $Errno2 has a fallback case, $Errno3 can now be coerced into $Errno2. All cases of $Errno3 not present in $Errno2 will be coerced to $Errno2s fallback case: "unknown". This does mean data might be discarded during coercion, however I'd argue this is OK since you'd have to explicitly op-in to this behavior in the original type.

Restrictions:

• Only constructor-less cases can be marked as fallback, ie cases that carry no additional data.
• $Errno3 must be a superset of $Errno2.

[lukewagner]

That's a really great point! I think there's also a symmetry argument in favor of this: in the same way that records (products) are the dual of variants (coproducts), this idea seems like the dual to optional record fields (which, the idea has been, subtyping would take into account, allowing a record subtype to omit optional fields of a record supertype).

One modest tweak on this idea is: instead of marking one case as the fallback in the supertype, we could instead allow individual cases to declare that they are a "refinements" of other, more-general cases such that, if the supertype doesn't know about a subtype's refined case, the refined case can be implicitly coerced to a general case that the supertype does understand. E.g.,:

(variant (case "one") (case "two" (refines "one")))

could be a subtype of

(variant (case "one"))

with two cases being coerced into one cases. (IIUC, the subtyping relation would also require that the payload type of the refined case was a subtype of the payload type of the general case.) The nice thing here is that we didn't need any forethought when defining the original (super) type -- we got to add refinements purely retroactively. Also, this allows different refined cases to generalize to different general cases (so, e.g., I could refine both the original success and failure cases to refined cases of each).

WDYT?

[badeend]

I think that's an even more flexible solution. Nice!

IIUC, the subtyping relation would also require that the payload type of the refined case was a subtype of the payload type of the general case

Yes, with the caveat that the proposal currently doesn't mention Unit types and their coercion rules. I.e. can an s32 be coerced into nothing? That would be required for variants like this:

(variant (case "one") (case "two" s32 (refines "one")))
(variant (case "one"))

[lukewagner]

Yep, that makes sense.

[rossberg]

That's a really great point! I think there's also a symmetry argument in favor of this: in the same way that records (products) are the dual of variants (coproducts), this idea seems like the dual to optional record fields (which, the idea has been, subtyping would take into account, allowing a record subtype to omit optional fields of a record supertype).

Concretely, the duality is between allowing dropping optional fields in a product and allowing adding cases in an optional sum. That is:

(record (field A)) <: (record (field A) (field (opt B)))
(opt (variant (case A) (case B)) <: (opt (variant (case A)))

FWIW, these are the rules we use in Dfinity's Candid IDL. (The opposite directions hold by standard subtyping.)

The suggested fallback is another way to express the same option in a more specialised manner.

As @lukewagner points out, you'll need the foresight to add that option initially if you want to make a variant "extensible". The refinement idea kind of avoids that, but I suspect that in practice, you'll probably still need some foresight to add a custom fallback case to refine with future cases, because there isn't always an appropriate "regular" case to refine, and the subtype requirement might also get in the way.

[badeend]

you'll need the foresight to add that option initially if you want to make a variant "extensible". The refinement idea kind of avoids that, but I suspect that in practice, you'll probably still need some foresight to add a custom fallback case to refine with future cases, because there isn't always an appropriate "regular" case to refine

Totally agree. To take it even one step further: I bet that when a variant is designed to be extensible, in practice all other cases will be refinements. Continuing my initial example:

(variant
    (case "unknown")
    (case "access" (refines "unknown"))
    (case "badf" (refines "unknown"))
    (case "busy" (refines "unknown"))
    (case "again" (refines "unknown"))
)

the subtype requirement might also get in the way.

Was this remark specific to finding a fallback after the fact. Or do you have subtyping concerns in general?

---

Nitpick; the term "refine" might confuse people into thinking it has something to do with refinement types. Alternatives: "extend", "specialize", "enhance", ...

---

Would you be open to adding this functionality to Interface Types in some form or another?

[lukewagner]

Maybe defaults-to instead of refines?

In any case, yes, I think this makes sense to add in some shape or form in the same spirit as the record optional field subtyping.

[badeend]

For posterity: work is being done in https://github.com/WebAssembly/component-model/pull/6

[badeend]

Closing this, since the PR has been merged.