Experiment - Class annotation for disallowing use as a super-type, a.k.a "sealed classes"

[srawlins]

Users have expressed a desire for the ability to mark a class as "final" or "sealed," so that it is unavailable as a super-class. That is, it is an error for a class to extend, implement, or mix in a "sealed class." Specifically, the Angular Dart team wishes to seal certain classes that have very undefined behavior when users subclass.

There is good discussion on an MVP, make-something-cheap-available-to-users request for a @sealed annotation.

An experiment

All that is being suggested here, after some discussion ¹, is an annotation in the meta package, @sealed, enforced with some analyzer Hint codes.

Use cases

The primary use case is to remove burden from, and give assurance to, authors of public classes. Today, as all classes are "open," there is an expectation from users that they can extend, implement, and mix in classes from other packages safely. For this to be true, authors must actually have extensions, implementations, and mix ins in mind, unless they write "DO NOT EXTEND/IMPLEMENT" in their Documentation comments.

A "sealed classes" feature can remove burden from authors by allowing for a class that doesn't need to support the possibility that other classes use it as a super-class. Authors can write a class

• whose methods call other public methods,
• whose methods refer to private members,

without worrying about how to support sub-classes.

A "sealed classes" feature can give assurance to an author, when considering how a change to the class will affect users. An author can:

• change a method to refer to a public method,
• add public methods,

without worrying about how the change will affect potential sub-classes.

Definition

A sealed class, one which has been annotated with @sealed from the meta package, can only be extended by, implemented by, or mixed into a class declared in the same "package" (to be defined) as the sealed class. The consequences of "sealing" a class are primarily in static analysis, and would be implemented in the analyzer package:

• It is a Hint if the @sealed annotation occurs on anything other than a class.
• Given a class C declared in a package P,
  • It is a Hint if a class which is declared in a package other than P extends, implements, or mixes in C.
  • It is a Hint if a class, declared in P, and either extending, implementing, or mixing in C, is not also annotated with @sealed.

Why a Hint?

All analyzer codes that do not arise from text found in the spec are HINT codes (well, except TODO and LINT, and STRONGMODE* codes). Since this is the most formal proposal for an annotation that I know of, perhaps it will pass enough muster to be listed as an ERROR or WARNING... if there is such a request, this can be specified at such time. (@MichaelRFairhurst requested below not to use error.)

Ignorable?

All analyzer codes are ignorable with an // ignore comment. The above Hints will also be ignorable:

• // ignore: USE_OF_SEALED_SUPER_CLASS will allow a sealed class to be used as a super-class outside of the library in which it is declared.
• // ignore: OPEN_SEALED_SUB_CLASS will allow a non-sealed class to sub-class a sealed class from within the library in which each is declared.

(@MichaelRFairhurst requested below to allow these analyzer results to be ignorable.)

Alternative definitions

Library-level sub-classes

Library-level sub-classes would be very similar to package-level subclasses, but would be more restrictive. Library-level sub-classes make an earlier suggestion of performance experiments easier, but the performance experiments are now a non-goal of this annotation. Members of the Dart team feel that a package boundary is the more natural boundary for something that authors create for themselves; typically the authors of a package "own" the whole package, rather than distinct pieces.

Additionally, new visibility mechanisms were suggested; maybe Dart can support an "open part" (as @eernstg suggests below or "friend library" (as I suggest in a comment on testing private methods). The part/friend concept would help back-ends close the list of all sub-classes, but we don't have this concept yet, so cannot experiment yet.

Single concrete class

The "sealed classes" feature originally restricted sealed classes to be un-extendable, un-implementable, and unable to be mixed in, by any class anywhere ("final classes"). @eernstg argues below that the reasons for making a "final class" are different from those for making a "sealed class," and that it would not be very meaningful to switch the definition of @sealed from one to the other.

Since Angular Dart can make use of library-sealed just as easily, and back-ends like the VM can optimize just as easily, then we use the library-sealed definition.

Isn't "experiment" just another word for "I don't want to go through the trouble of actual approval?"

We actually do want to experiment. Real world usage can help the language team steer towards correct choices later:

1. Are sealed classes useful?
2. Do public class authors "abuse" sealed classes? ("Abuse" might mean "very defensively program with." We want to see how sealed is used.)
3. Do users // ignore the "sealed classes" Hints?
4. Do users fork sealed classes in order to unseal them?
5. Is the "sealed" concept useful as the single great class modifier affecting sub-classing? Do users basically use it as "final?"

Depending on the answers to the above, "sealed classes" may be shelved, or implemented with the same definition as the annotation, or may be implemented with changes. Other features may be implemented or experimented with, such as final classes, sealed-by-default, open, noImplicitInterface, etc.

Cost of rolling back the experiment

@munificent points out below that asserts-in-initializers and supermixins were both "experiments" that did not smoothly transition to full support; we should try to avoid a similarly bumpy road.

If the @sealed experiment "fails", i.e. the team decides it should be rolled back, it can be done so without much fanfare. Rolling back the feature means removing enforcement from the analyzer (and any back-ends with performance experiments based on the annotation). A Dart release will include a release note saying something to the effect of "Any sealed classes may now be sub-classed at will; don't rely on them not being sub-classed."

Path towards a language feature

For package-level "sealed classes" to graduate from an experimental annotation to a language feature (like a class modifier), a definition for package will first need to enter the language. There is currently no effort towards defining such a thing, but there is motivation from several facets of Dart to make one.

Prior art, discussion

Java

A case of prior art / prior discussion, Joshua Bloch, once Google's chief Java architect and author of Effective Java, wrote in Effective Java,

Design and document for inheritance or else prohibit it.

Joshua goes on to explain the work involved in properly designing, implementing, and testing a class that is subclassable, which is substantial. When Joshua writes "or else prohibit it," he is referring to the use of either (a) marking a class final or (b) using only private constructors. The private constructor solution, (b), does not work for Dart today, as all classes have implicit interfaces which can be implemented. A "no implicit interface" modifier could be a sibling feature to this "sealed classes" feature, but I consider it far out of scope.

Kotlin

Kotlin has a more advanced feature set regarding "sealing" or "finalizing" classes. Here's a quick rundown:

• By default, a class is "final," such that it cannot be subclassed. This is directly motivated by Joshua Bloch's writing. To make a class open for subclassing, the open modifier must be applied.
• A class can be "sealed" with the sealed modifier. This means that the class can only be sub-classed within the file where it is declared. This has the effect that the author immediately knows all of the possible direct sub-classes, and can use this knowledge in switch statements; you can cover every possibility of sub-classes with a finite and immediately known set.
• One more note: the sub-classes of a sealed class have no restrictions different from that of other classes; they can be marked as open, and they can be sub-classed when open.

I really like these two similar but distinct features. For a sealed class, the ultimate set of concrete classes with a sealed class as a super-class cannot be known, unless all direct sub-classes are "closed." This property is under the author's control; if the author just wants to know all direct sub-classes, for use in, e.g. a switch statement, (and if they're willing to support the idea of sub-classing), then they can mark sub-classes as open. Otherwise, if they want to know every concrete class with a sealed class as a super-class, they can leave sub-classes closed, and not have to support the concept of sub-classing.

Other languages

Other languages, in addition to Java and Kotlin, have a similar / identical feature, either "sealed" or "final" classes, including C++, C#, and Swift. Neither JavaScript nor TypeScript have a similar feature.

Footnotes

¹ Initially, I did not predict the level of discussion that this feature request would raise. Initially, I thought that the experimental @sealed annotation would land quickly and quietly. This feature request was initially for a language feature, "sealed classes." After seeing all of the issues being raised, and some thinking that this was just a proposal for the experimental annotation, I've decided to make it that.

[munificent]

@srawlins, I like your updates to the proposal and I think it's reasonable to move forward with doing an experiment with @sealed having library-level granularity. 👍

I think it would be useful if you could extend that scope to some sort of larger "package" or "compilation target" granularity. But that notion doesn't currently exist, and would likely take a lot of effort to define. I don't think we should block @sealed on that.

[leafpetersen]

Thanks a lot for all of the useful input folks, I've picked up a lot from the discussions here and offline - I appreciate everyone staying civil and constructive on a topic that I know there are strong feelings about. And thanks very much to @srawlins for writing up a detailed proposal and spending time on feedback. I'm still digesting this a bit, but it seems like the interesting issues are:

• Granularity
  • package, library, or both?
  • what's a package?
  • How to balance the needs of tests against the desire to seal things
  • How to balance the needs of package authors and consumers given that:
  • package authors want to be able to make breaking changes
  • package consumers (may) want to be able to make unexpected use of libraries
  • the pub ecosystem relies on authors and consumers actually respecting semver

On the question of @sealed vs @packageSealed: for the experiment would it be reasonable instead to have @sealed take some kind of optional argument defining the scope? This might give us data on what scope authors actually want? Just a thought.

[srawlins]

Some of us talked offline more, and we've changed gears to recommending a package-level defintion of "sealed classes." I've updated in the summary up top, the Definition, Alternative Definition, and Path towards a language feature sections above.

This was largely the result of dropping the performance experiment from the story; although sealed classes could still be performance-experimented, by using sealed classes that are sealed within libraries, or by using package-level compilation units in something like dynamic code loading. In any case, performance considerations are just not part of this feature request.

@eernstg, you strongly recommended using library-sealed classes over final classes in sealed's clothing. I don't see why package-sealed would change your argument, but please comment if it does.

[eernstg]

No problem!

I just argued that it's more useful to have a notion of sealed classes which creates a closed set of types, rather than just sealing one class at a type (and being unable to close any type hierarchy as a hole, because no non-bottom types can be 'sealed' when that just means final).

Any granularity which is a library or something bigger (a set of libraries that somehow form a group) would have that property. Any developer who is working on, say, a package which contains a sealed type hierarchy would still be able to work on the complete class hierarchy and rely on the fact that there are no other subtypes. So that developer would still know that no "outside" subtypes are going to break if something changes, it's enough to worry about breaking changes as seen from clients (and private classes would always be invisible and hence safe when the scope is at least a library).

It would always be possible for an implementation to use the improved situation where something is sealed in a library to obtain some additional optimizations, and it shouldn't be hard to support library-sealed classes later on.

[lrhn]

I, personally, have no problem adding "package" as a concept to the language. It will be "Dart package", because that's all the source can represent. I can see (multiple) ways to do "package private" with that, and perhaps even allow overriding it for testing. So, making sealed package-scoped is not necessarily a blocker for making it a language feature.

That said, I'd much rather be working on features that make it possible to modify a class without breaking sub-classes, rather than require you to seal the class up-front just to have the ability to maybe modify it later. Such a feature would help you even if you didn't seal the class.

I can absolutely guarantee you that if we had had sealed in the language then, Future would have been sealed from the start. That would disallow all the current classes implementing Future, but the one implementation would be faster and easier to modify.

[nodinosaur]

I know that this thread is really about the lack of Sealed Unions but and an interim maybe Dart Sealed Unions and could be useful for some of you.

Out of frustration not having Kotlin style Sealed Unions for Java, JavaSealedUnions was created and have I successfully used it in many Android projects. Dart Sealed Unions is a one-to-one port of this and I am now able to successfully use this in my Flutter projects.

I am hoping that this will help some until we have a real implementation.

@brianegan has a sample MVi architectural example that uses this package.

[srawlins]

This has been released in Dart SDK release 2.1.1; the CHANGELOG notes the two new codes: SUBTYPE_OF_SEALED_CLASS and MIXIN_ON_SEALED_CLASS.