proposal: Go 2: sealed types

[deanveloper]

Related: #43123, #28987, #28939

Problems

This proposal aims to solve two problems:

1. Some types require instantiation with constructor, whether it is to set private fields, or initialize certain fields which may make the structure not very useful (ie nil maps).
2. Some types are meant to act as an enum, especially those which are declared via iota. Then validity of variables of these types need to be checked at runtime, when ideally these checks should never need to happen.

The issue with these both reside in the fact that types are allowed to be instantiated outside of the package in which they were declared in, allowing for the structure to have any shape. This means that some form of runtime validation must always be done.

Solution: Sealed Types

This is very similar to #43123, although it permits a few more actions to be done.

Usage

// chess/piece.go
package chess

type Piece sealed int

const (
    PieceNone Piece = iota
    PiecePawn
    PieceRook
    PieceKnight
    PieceBishop
    PieceQueen
    PieceKing
)

// engine/main.go
package main

import ".../chess"

func main() {

    var myPiece chess.Piece // legal
    myPiece = chess.PiecePawn // legal

    myPiece = 5 // illegal
    myPiece = chess.PieceRook + 1 // illegal
    myPiece = chess.PieceRook + chess.PiecePawn // illegal

}

Grammar Changes

- TypeDef = identifier Type .
+ TypeDef = identifier [ "sealed" ] Type .

Reflection considerations

A new method in reflect.Type: Sealed() bool which returns whether the type is a sealed type.

Sealed types may not be converted to via reflection (ie reflect.Convert).

reflect.Value.CanSet() bool should return false on sealed types, and on elements (ie fields, slice/array elements, or any recusive step thereof) of sealed types.

Behavior

When a Type is preceded by "sealed", this type becomes a sealed type. Inside the package that they are declared in, sealed types behave normally as any other type would. However, when used outside of the package they are declared in, values may only be created via copying, language-provided zero values, or from other packages.

Types which are based on these sealed types are not themselves sealed. For instance, consider type MyType other.PCT where other.PCT is a sealed type. MyType would not be a sealed type. However, aliases of sealed types are treated as if they were created in the other package.

This means that outside of the package that they are declared in, sealed types may not:

1. Be converted to
   • x := other.PCT(y)
2. Be converted from an untyped constant
   • var x other.PCTInt = 5
3. Be declared using composite literals
   • x := other.PCTStruct{...}
4. Be converted via reflection
   • value := reflect.Convert(value, otherPCTType)
5. Be used with mathematical operators
   • x := somePCTInt + 1
   • x := 5 * somePCTInt

However, even outside of the package they are declared in, sealed types may:

1. Be returned from a function outside of the current package
   • var x other.PCT = other2.ReturnsPCT()
2. Be copied
   • var x other.PCT; y := x
3. Be modified via an unsafe pointer
   • var x *myType := *myType(unsafe.Pointer(&someOtherPCT))
4. Be created via zero-value generation
   • var x other.PCT
   • x, _ := notOtherPCT.(other.PCT)
   • x, _ := stringToPCTMap["not in map"]
5. Be asserted to
   • x := myInterface.(other.PCT)
   • x := reflectValue.Interface().(other.PCT)

Unmarshaling considerations

Default unmarshallers should avoid unmarshalling to any sealed types which do not implement an interface some kind of Unmarshaller interface (ie: json.Unmarshal should fail for sealed types which do not implement json.Unmarshaller). This is also enforced by the fact that reflect.Value.CanSet will return false, so it's not like the encoding/json package would be able to unmarshal easily anyway.

Open questions

1. sealed outside of type declarations. This doesn't seem to make a lot of sense so I am tempted to make this illegal. For instance, the function declaration func Increment(i sealed int) int cannot be called, as one would have to convert int to sealed int which is illegal.

[deanveloper]

Would you consider yourself a novice, intermediate, or experienced Go programmer?

• Intermediate

What other languages do you have experience with?

• Java, C, Python, JavaScript, Kotlin, C#

Would this change make Go easier or harder to learn, and why?

• It would make Go harder to learn. Sealed types are an additional feature that must be learned by users of a package that uses sealed types. However hopefully error messages are clear enough to teach users about sealed types.

Has this idea, or one like it, been proposed before?

• Yes, in #43123

If so, how does this proposal differ?

• It is a bit more permissible, as well as more details as to how some interactions with other language features work.

Who does this proposal help, and why?

• This helps library writers ensure that data is valid at compile-time.

What is the proposed change?

• above

Please describe as precisely as possible the change to the language.

• above

What would change in the language spec?

• above

Please also describe the change informally, as in a class teaching Go.

• above

Is this change backward compatible?

• No, it introduces a new keyword sealed. However if we do not wish to introduce this new keyword, using package instead (ie type Piece package int)

Breaking the Go 1 compatibility guarantee is a large cost and requires a large benefit.

• I believe the benefit of having constructor-only types as well as compile-time enums is worth a reserved word. Although if it is decided that these features are not worth breaking Go 1 compatibility, package may be used instead (as well as phrasing in "reflect" package, etc)

Show example code before and after the change.

• above

What is the cost of this proposal? (Every language change has a cost).

• A bit of cognitive overhead, more compile-time errors, and slightly slower compile-times.

How many tools (such as vet, gopls, gofmt, goimports, etc.) would be affected?

• Likely all of them, as a new keyword is being added. gopls will definitely need some changes, and gofmt may need some changes for the TypeDecl syntax changing.

What is the compile time cost?

• I do not know much about the Go compiler, although the typechecker will have to keep track of which types are sealed and block some operations on sealed types.

What is the run time cost?

• Quicker runtime for usages of enums and types intended to be used with a constructor, as less needs to be verified at runtime. However all calls to CanSet() do now need an additional Sealed() check.

Can you describe a possible implementation?

• above

Do you have a prototype? (This is not required.)

• no

How would the language spec change?

• above

Orthogonality: how does this change interact or overlap with existing features?

• It doesn't overlap with much, however it does interact with iota which is often used with enum types.

Is the goal of this change a performance improvement?

• no

Does this affect error handling?

• no

Is this about generics?

• no

[ianlancetaylor]

In general Go has a simple type system, and encourages programming by writing code, rather than programming by writing types. This seems like a step away from that.

That said, you can get similar benefits by using a struct with unexported fields. What advantages do sealed types provide over that approach?

[deanveloper]

I largely agree that we shouldn't be programming by writing types. Perhaps using this in order to require the use of a constructor wasn't great, although it certainly helps self-document that the type should not be used on its own. However for enums this provides a huge benefit. One of the large reasons that #28987 ("enums as an extension to types") wasn't ideal was that it tried to tie several features (named values, compile-time validation, and a way to define the values) all into a single feature. Instead, those three features should be implemented as three separate features that work harmoniously with each other.

Specifically, this proposal attempts to solve the "compile-time validation" portion of what we may want in order to have traditional enums.

[earthboundkid]

Not being able to do math with enums would make it impossible to do things like bitfields for perm.Read | perm.Write.

[deanveloper]

It is not impossible. First of all - bitfields are already possible in Go today with very little runtime validation being needed, as it is common behavior that bits which get set and are not listed by the package are discarded. Sealed types don't particularly affect bitfields in any way. However, if a bitfield does in fact need validation that it only used constants provided, one may use a function instead:

// FieldUnion returns the binary OR of all fields provided.
func FieldUnion(fields ...Field) Field {
    var union Field
    for _, field := range fields {
        union |= field
    }
    return union
}

This function is provided by the package, so it is allowed to use the union operator. This however removes the ability to calculate the union of the fields at compile-time which is a bit unfortunate.

While I am tempted to make an exception for | operator, I don't think it's the right thing to do. I think in the case of bitfields, it is better to simply not use sealed types.

[Dynom]

We use Enums extensively. We're sharing data/state produced by different technologies and have many Enums with values out of our control. A sealed type would offer additional confidence which would otherwise be quite convoluted to achieve.

However this is possibly a niche situation that might not warren a language change, especially since proper etiquette and/or code-review policy should catch the practices from the OP example.

[ianlancetaylor]

The proposal says that one of the problems to be solved is:

Some types require instantiation with constructor, whether it is to set private fields, or initialize certain fields which may make the structure not very useful (ie nil maps).

But this proposal permits creating zero values of sealed types, and it permits assignment of sealed types. So it doesn't seem to solve this problem.

It's not clear how a sealed type would be used other than for enum-like types for which 0 is a valid enum value.

[deanveloper]

But this proposal permits creating zero values of sealed types, and it permits assignment of sealed types. So it doesn't seem to solve this problem.

Whether or not assignment to elements in a composite type is allowed is an open question. However, assuming that it isn't (which is what I suggest) there is no way to modify the values inside of a composite type. This means that the only values which can exist are the zero value, and values created by the package. This should solve the problem of creating a type that requires a constructor, as people cannot modify the components of the type on their own.

I have also realized while writing this comment that not allowing to modify values of composite types essentially makes this a form of a read-only types proposal (without the complex interactions of assignability). If those ever come to Go (not sure that they will) then it would be a bit complicated to figure out the logic between read-only and sealed types. Not sure how much I like that.

Also I am not sure how sealed would work outside of a type declaration, ie func Increment(i sealed int) int, it would seem that it is impossible for Increment to be called from outside the package (except with a zero value). Perhaps this should be illegal? Adding this as another open question.

[ianlancetaylor]

I agree that this proposal doesn't permit modifying values inside a composite type. However, one of the natural uses for a type that requires instantiating with a constructor is a type that contains a lock, as such types can't be copied by value (that's why the vet tool has a copylocks check, to look for this case). It would seem natural to use sealed types for this, but it won't work, because sealed types can be copied by value.

I'm stressing this because it seems to me that this is one of the first things that comes to my mind when I read this problem statement, but the proposal doesn't actually avoid that problem.

[creker]

I feel like this kind of defeats the purpose of the proposal - "sealed types may be created via zero-value generation". If we can instantiate zero-value sealed types then we are back to square one - either package needs to support zero values or make it programmer's responsibility to use it correctly.

I know it's ugly but what about restricting sealed types to pointers only (something similar was mentioned in the other proposal)? That way zero-value becomes nil pointer which will be less usable and in most cases would cause panics. We can go even further and force panics on function calls with nil pointer receiver of a sealed type. That makes sealed types very special and nothing like anything else in the language but at least it gets us closer to the desired behavior. But I feel like no matter what we do here Go's type system is too simple for this. Ideally we would want proper constructors to solve this.

[deanveloper]

@ianlancetaylor

I agree that this proposal doesn't permit modifying values inside a composite type. However, one of the natural uses for a type that requires instantiating with a constructor is a type that contains a lock, as such types can't be copied by value (that's why the vet tool has a copylocks check, to look for this case). It would seem natural to use sealed types for this, but it won't work, because sealed types can be copied by value.

In my personal experience I typically have troubles with reference types. I would like each type to have its own copy of the passed-in slice, so the way I typically enforce this is with a constructor function which makes clones of the slices/maps which are passed in. I did not think about mutexes when making this proposal, however I believe forcing the use of a constructor is still useful because it can initialize a pointer to a mutex, unless ther is a problem with that in which I am missing.

edit - I understand now the issue with allowing zero values in combination with mutexes. Unfortunately I don't think there is really a good way to resolve this, however the copylock check from go vet should still work in relieving this trouble. I would recommend maybe a separate type modifier which prevents the copying of a type, however then we start getting into the issue of "programming with types", as well as seeing very verbose type definitions (like type SyncMap[K comparable, V any] copylock sealed struct { ... }). It would be very nice for Go to have some kind of annotation system similar to struct tags which may help remedy this.

edit 2 - after thinking about it a bit more, a good way to solve that issue is to use pointers to mutexes (as mentioned before) and initialize it when a method is called on the type. Because the struct is sealed, it can only be written to via methods.

@creker

I know it's ugly but what about restricting sealed types to pointers only (something similar was mentioned in the other proposal)? That way zero-value becomes nil pointer which will be less usable and in most cases would cause panics.

Going back to pointers doesn't actually help anything. Any time where we'd encounter an error with using zero values, we'd encounter errors with using pointers as well. Library writers would be inconveniencing users to need to deal with pointers everywhere with little benefit. Without zero values...

We can't conditionally initialize a value

var foo other.SealedType
if someCondition {
    foo = fetchFromResource(...)
} else {
    foo = fetchFromOtherResource(...)
}

// workaround...
var tempFoo *other.SealedType
if someCondition {
    temp := fetchFromResource(...)
    tempFoo = &temp
} else {
    temp := fetchFromOtherResource(...)
    tempFoo = &temp
}
foo := *tempFoo

It is literally impossible to check if an interface is of our sealed type

// what is `val` when `ok` is false?
val, ok := someInterface.(other.SealedType)

We can not use our type in any kind of composite type (without using pointers which, as shown earlier, inconveniences users of the package, as well as not adding any safety benefit)

slice := make([]other.SealedType, 2) // slice[0]
myMap := make(map[string]other.SealedType) // myMap[""]
ch := make(chan other.SealedType) // <-ch on a closed channel
myStruct := struct { st other.SealedType }{} // myStruct.st

In conclusion - zero values are necessary. I wholeheartedly believe that if we didn't allow zero values on sealed types, countless users would ask library writers to stop using sealed types because they would be so inconvenient to use.

[creker]

@deanveloper

Any time where we'd encounter an error with using zero values, we'd encounter errors with using pointers as well.

There's an important difference. Non-pointer zero values are silent. They're in invalid state but otherwise behave like nothing's wrong. Only when something catastrophic happens or the code explicitly checks for invalid state will the user get an error. Or not, there's no telling. With pointers, especially if we force panic on method calls, the code will explode immediately. That's much more desirable behaviour that solves what this issue actually aims at - prevent users from using incorrectly initialized types.

We can't conditionally initialize a value

We can, just force users to use pointers or make pointer semantic implicit like it is with channels and maps.

It is literally impossible to check if an interface is of our sealed type

I don't understand this. In your example val will be nil. There's nothing special about pointer-only sealed types. They would behave like any other struct with pointer receiver.

We can not use our type in any kind of composite type

We can. Sealed types would be all nil.

[deanveloper]

I see what you mean now, I assumed you had meant that we would just make the zero value of nil types invalid (as the previous proposal had stated), and we would instead just force users to always use pointers to the value if we want to put it in maps and such. This would cause my scenarios to fail.

Essentially you would want all sealed types to be reference types then? I'll have to think about some of the implications of doing that. It'd definitely be strange, but it might work out okay since the package's consumers can't mutate the value. That may also solve the mutex problem that @ianlancetaylor mentioned. It would have some strange behavior for package writers, though.

[ianlancetaylor]

This is an interesting idea but it has too many awkward edges. Based on the discussion above, this is a likely decline. Leaving open for four weeks for final comments.

(It may be worth considering whether #6386 would permit you to do what you want, by using exported const struct values. That might be similar to this idea, I'm not sure. Though of course that proposal has not been accepted either.)

[ianlancetaylor]

No further comments.