golang / go

The Go programming language
https://go.dev
BSD 3-Clause "New" or "Revised" License
124.07k stars 17.68k forks source link

proposal: spec: generics: type initialization with value parameter #67064

Closed smyrman closed 1 month ago

smyrman commented 6 months ago

Go Programming Experience

Experienced

Other Languages Experience

C, Python

Related Idea

Has this idea, or one like it, been proposed before?

Yes

Typed enum support has been suggested in https://github.com/golang/go/issues/19814. That proposal has a overlapping use-case, which is to allow conversion to/from enumerated values and strings.

This proposal is different in that it's not limited to enums; it's about type parameterization using values. This proposal could be used to define an enum package (in or outside) the standard library to aid with enum conversion and validation, but it's not primarily a proposal for implementing enums.

Const parameters has been suggested in https://github.com/golang/go/issues/65555. That proposal aims at allowing to create types using constant parameters (as an alternative or addition to typed parameters).

This proposal is different, and perhaps arguably worse, in that it isn't limited to constant parameters. The proposal is also different in the sense that it's aiming for a clear separation between type and value parameters.

Does this affect error handling?

No

Is this about generics?

Yes, it's about allowing type initialization to refer to a value.

Proposal

Sometimes, it would be useful to initialize a type that is initialized not only using type parameters, but also value parameters. This allow generic code to be written for multiple use-cases, such as:

Enum use-case

Goal: define an enum like value with minimal boiler plate (library code is allowed).

Current solution (no language change)

Library:

package enum

// Lookup provides a two-way lookup
// between values of type T and strings.
type Lookup[T comparable] interface{
   Value(string) (value T, ok bool)
   Name(T) (name string, ok bool)
}

func MustCreateLookup[T comparable](map[T]string) EnumLookup{
    // implementation doesn't matter.
})

Application:

type MyType uint

const (
    _ MyType = iota
    MyTypeValue1
    MyTypeValue2
)

var myTypeLookup = NewEnumLookup[MyType](map[MyType]string{
   MyTypeValue1: "value1",
   MyTypeValue2: "value2",
})

func (v MyType) Value() (driver.Value, error) {
    v, ok := myTypeLookup.Name(v)
    if !ok {
        return fmt.Errrof("unrecognized value: %v", v)
    }
    return v, nil
}

func (v *MyType) Scan(src any) error {
     s, ok := src.(string)
     if !ok {
         return fmt.Errorf("incompatible type: %T", src)
     }
     _v, ok := myTypeLookup.Value(s)
     if !ok {
         return fmt.Errrof("unrecognized value: %v", s)
     }
     *v = _v
}

Suggested solution (language change)

Library:

package enum

// Lookup provides a two-way lookup
// between values of type T and strings.
type Lookup[T comparable] interface{
   Value(string) (value T, ok bool)
   Name(T) (name string, ok bool)
}

func MustCreateLookup[T comparable](map[T]string) EnumLookup{
    // implementation doesn't matter.
})
package enumsql

type Enum[L enum.Lookup[T], T comparable][lookup L] T

func (v Enum[L,T][lookup]) Value() (driver.Value, error) {
    v, ok := lookup.Name(v)
    if !ok {
        return fmt.Errrof("unrecognized value: %v", v)
    }
        return v, nil
}

func (v *MyType) Scan(src any) error {
     s, ok := src.(string)
     if !ok {
         return fmt.Errorf("incompatible type: %T", src)
     }
     v, ok := myTypeLookup.Type(s)
     if !ok {
         return fmt.Errrof("unrecognized value: %v", s)
     }
     *v = _v
}

Application:

type MyType = sqlenum.Enum[uint](enum.MustCreateLookup[MyType](map[MyType]string{
   MyTypeValue1: "value1",
   MyTypeValue2: "value2",
})

const (
    _ MyType = iota
    MyTypeValue1
    MyTypeValue2
)

Matrix use-case

An attempt of supporting what's described in https://github.com/golang/go/issues/65555. There could be reasons in which this is harder than the first case. I am not sure it's a priority use-case.

Current solution (not type safe)

type Matrix[T any] struct{
    data []T
    w, h int
}

func (m Matrix[T]) Index(x, y int) (v T, bool) {
    if 0 < x || x >= w {
        return v, false
    }
    if 0 < y || y >= h {
        return v, false
   }
    return m[x+y*w], true
}

func NewMatrix[T any](w, h int) Matrix {
    if w <= 0 || h <= 0 { panic("invalid size")}
    return Matrix{data: make(w*h), w: w, h: h}
}

Solution with language change (type safe)

func NewMatrix[T any](n, m int) [n][m]T {
    return [n][m]T{}
}

Alternatively, if it's easier for implementation reasons:

func NewMatrix[T any][n, m int]() [n][m]T {
    return [n][m]T{}
}

Language Spec Changes

To be decided; early draft

Type value parameter declarations

A type value parameter list declares the type value parameters of a generic type declaration; it can not be used by generic functions. The type value parameter list looks like an ordinary function parameter list except that the type parameter names must all be present and the list is enclosed in square brackets rather than parentheses. The declaration must come right after a type parameter list. If there are no type parameters, then an empty set of square brackets must precede the value parameter list.

Example (with a type parameter list prefix): [T any][v T] [][v string] [][_ any]

Just as each ordinary function parameter has a parameter type, each type value parameter has a type.

Informal Change

Just like a function can have both input parameters and output parameters, a generic type can have both type and type value parameters. Both parameters are used to generate a concrete implementation. Type parameters are replaced by specific types, while type value parameters are replaces by a reference to a specific variable reference. That is, if you where to generate code for it, you would typically see that instances of the type parameters are replaced by the proper type reference, while references to values, are replaced by references to package scoped variables.

A simple (and somewhat silly) example:

type Greeter[][v string] string

func (g Greeter[][v]) Greet() string {
   return fmt.Sprintf("%s %s!", v, g)
}

type Helloer = Greeter[]["Hello"]

// Hello Peter!
func main() {
  peter := Helloer("Peter")
  fmt.Println(peter.Greet())
}

Is this change backward compatible?

Yes

Orthogonality: How does this change interact or overlap with existing features?

It interacts with generic type declarations, by declaring a second parameter group.

Would this change make Go easier or harder to learn, and why?

I don't think it would make Go easier to learn. It's going to make it somewhat harder to learn generics in particular.

It might help deal with some pain-points if combined with the right standard libraries, such as easier enum handling.

Cost Description

Changes to Go ToolChain

Yes

Performance Costs

Unknown.

Prototype

I belie a ad-hock implementation could be written that depend on code generation.

Something similar to what was used for go generics, using .go2 or another file type format, could be used to scan the code and replace type value parameters with specific type implementations that replace the variable reference with a reference to a private package scoped variable; potentially one with a generated name.

I.e.

type Greeter[][v string] string

func (g Greeter[][v]) Greet() string {
   return fmt.Sprintf("%s %s!", v, g)
}

type Helloer = Greeter[]["Hello"]

would generatesomething like:

var _go2generated_Helloer_p1 string = "Hello"

type Helloer string

func (g Helloer) Greet() string {
   return fmt.Sprintf("%s %s!", _go2generated_Helloer_p1, g)
}

The proto-type could be limited not to allow usage of the new generics across package borders, and could run before compilation. It could also require explicit use of type aliases for declaring types.

A final implementations should not have such limitaitons.

ianlancetaylor commented 6 months ago

The general idea has merit. The specific syntax here doesn't seem great. We can't support

func NewMatrix[T any](n, m int) [n][m]T {
    return [n][m]T{}
}

That would appear to require us to compute types at runtime. But having two separate type parameter lists doesn't seem great either.

Also it's not clear that the use cases for this are strong enough to support the additional language complexity.

smyrman commented 6 months ago

A possible alternative syntax, could perhaps be prefix related. I.e. since we already support ~T to mean underlying type, perhaps we could have a different prefix to mean value assignment? Maybe =T could work?

// using = for value...
func NewMatrix[T any, N, M =int]() [N][M]T {
    return [N][M]T{}
}
ianlancetaylor commented 5 months ago

We could perhaps support generic parameters that are values. We wouldn't want to use =int for the syntax. But maybe we could write something like

func NewMatrix[T any, N, M const int]

Are there concrete use cases for this? The enum example doesn't seem to bring clear value. The matrix example might in principle be useful for SIMD, but would it really help in practice?

In particular, we don't want to add dependent types to Go. That is complexity beyond what we want in the language.

-- for @golang/proposal-review

smyrman commented 5 months ago

The use-case that made me think of this, was indeed the enum case. Or to be more specific, it's the use-case of translating between different enum representations in different layers of the application, and to minimize boiler plate when doing so by bake the functionality into a library (and through this proposal, maybe a language feature).

Let's do an illustrative example. Consider three representations for a data resource "Food" in three different layers (clean architecture style):

We will only look at the first two layers, as there is no fundamental difference between the DB layer and the API layer; the DB layer is just encoding the information into a different wire format then the API layer.

Let's start with the Business logic layer. The goal of this layer is to create a type that is agnostic to the encoding used in either the API layer (could have multiple maintained versions for backwards compatibility), or database (could have separate enum set for legacy reasons and/or multiple implementations).

package entities

type FoodType uint

const (
    _ FoodType = iota
    FoodTypeAubergine
    FoodTypePasta
    FoodTypeFish
)

type Food struct{
    ...
    Type FoodType
}

There is no text representation in this layer, as that's irrelevant to the business logic implementation.

For each API version a separate Food struct is defined, focusing on encoding to e.g. JSON. There is also a separate foodType lookup defined, including alias support for parsing legacy values or alternate spellings.

So in api/v1 we got:

package api

foodTypes = enum.NewLookup(map[entities.FoodType][]string{
    entities.FoodTypeAubergine: {"aubergine", "eggplant"},
    entities.FoodTypePasta: {"pasta"},
    entities.FoodTypeFish: {"fish"},
})

type Food struct{
    ...
    Type  textfield.Enum[foodTypes] // Implements TextMarshal / TextUnmarshal
}

func (ft *Food) FromEntity(e entities.FoodType) {
    ...
    ft.FoodType.V = e.FoodType
}

Assuming library code:

package textfield

type Enum[L var enum.Lookup[T], T comparable] struct{
    V T
}

func (e Enum[T,L]) MarshalText() ([]byte, error) {...}
func (e *Enum[T,L]) UnmarshalText(data []byte) error {...}

Comparing to a solution using only existing language features and with library helpers for encode and decode, you wouldn't save more than ~7 lines of code per enum, per API version. But it does add some cognitive overload.

Optimized example without new language features:

package api

foodTypes = enum.NewLookup(map[FoodType][]string{ // ~ changed
    entities.FoodTypeAubergine: {"aubergine", "eggplant"},
    entities.FoodTypePasta: {"pasta"},
    entities.FoodTypeFish:  {"fish"},
})

type FoodType structPentities.FoodType // + added

func (ft FoodType) MarshalText() ([]byte, error) { // + added
  return enum.MarshalText(foodTypes, ft) // + added
} // + added

func (ft *FoodType) UnmarshalText(data []byte) error { // + added
  return enum.UnmarshalText(foodTypes, ft, data) // + added
} // + added

type Food struct{
    ...
    Type  FoodType  // ~ changed
}

func (ft *Food) FromEntity(e entities.FoodType) {
    ...
    ft.FoodType = FoodType(e.FoodType) // ~ changed
}

So is it worth it? Probably not. But it's not without value. Especially as the count of models, enum-types and API versions adds up.

In real numbers and a real product (Clarify.io), I count at least 12 enum types in a single service in our backend code base. We are in the process of moving to clean architecture there. On enums alone, we could, with this language feature, save ~170 lines of boiler plate if we consider only one DB version and one API version. Considering other use-cases (not enum based), we can reduce another ~60 lines by removing a 3 line function from ~20 entity types.

Clean architecture is heavy on boiler plate, so places where we can reduce it, seams worth exploring.

DmitriyMV commented 5 months ago

Generic value parameters comes extremely useful when you want create a type with a static callback function. That is - currently you need to pass it to the type constructor and store It inside the struct. This is especially painful for something that accepts static functions as comparators.

Generic value parameters will allow you to write some generic type with sorting on top of the slice type without wrapping it in the struct.

smyrman commented 5 months ago

Just as a side-note... I think that if we where to allow const parameters only and not var, libraries could still do hacks to enable the var use-case:

func init() {
    lib.RegisterCallback(mykey, myCallback)
}

type MyType = lib.TypeWithCallback[myKey]
smyrman commented 3 months ago

I found another use-case that I want to highight, where a variant of this feature would be useful.

Recently, I have been trying out the pattern of encapsulated fields similar to what's used in ardanlabs/service. A basic example of what an encapsulated field is, can be found here. The essential portions of the linked code is provided here:

// Name represents a name in the system.
type Name struct {
    name string
}

// String returns the value of the name.
func (n Name) String() string {
    return n.name
}

var nameRegEx = regexp.MustCompile("^[a-zA-Z0-9' -]{3,20}$")

// ParseName parses the string value and returns a name if the value complies
// with the rules for a name.
func ParseName(value string) (Name, error) {
    if !nameRegEx.MatchString(value) {
        return Name{}, fmt.Errorf("invalid name %q", value)
    }

    return Name{value}, nil
}

Reasons for using such a field in a (business layer) model, are similar to the reasons of using time.Time over a string or int64. By placing the underlying value in a private struct field, we ensures that the field is either valid or empty, simplifying the business logic for working with the model later on.

Here is an example model using encapsulated fields:


type User struct {
    ID             xid.ID
    Active         bool           // doesn't need encapsulating; already guaranteed valid
    Name           Name           // encapsulated string
    Email          Email          // encapsulated string
    ProfilePicture ProfilePicture // encapsulated string
}

How value parameterization could help

With this type of programming, I end up with many similar fields with slight differences. While this is an expected trade-off, it also means that this could be a candidate for generics. Usually there are categories of field types that could be initialized with a variable to reduce boiler plate. For most cases this variable could be const (with some trade-offs). That is, we may be interested in a variable, but a const will usually allow us to get such a variable from somewhere.

For the case above, we could picture:

type PatternString[p const string] struct {
    value string
}

func (n PatternString[p]) String() string {
    return n.value
}

func ParsePatternString[p](value string) (PatternString[p], error) {
    if !regex.MustCompile(p).MatchString(value) { // could be optimized, but this works.
        return PatternString[p]{}, fmt.Errorf("value %q does not match pattern %q", value, pattern)
    }

    return PatternString[p]{value}, nil
}

With initialization:

const patternName = "^[a-zA-Z0-9' -]{3,20}$"

var _ = regexp.MustCompile(patternName) // ensure the pattern is valid.

type Name = PatternString[patternName]

func ParseName(name string) (Name, error) {
    return ParsePatternString[patternName]
}

I could now add Email, ProfilePicture and any other fields that can be validated via a regular expression, from the same underlying type.

type Email = PatternString[patternEmail]

func ParseEmail(name string) (Email, error) {
    return ParsePatternString[patternEmail]
}
type ProfilePicture = PatternString[patternProfilePicture]

func ParseProfilePicture(name string) (ProfilePicture, error) {
    return ParsePatternString[patternProfilePicture]
}

Another variant:

// let the proto-type be private
type patternString[p const string] struct {
    value string
}

Use inheritance so that a filed can be extended:

type ProfilePicture struct{
    patternString[patternProfilePicture]
}

func ParseProfilePicture(name string) (ProfilePicture, error) {
    return ProfilePicture{parsePatternString[patternProfilePicture]}
}

func (p ProfilePicture) URL() url.URL {...}

func (p ProfilePicture) PreviewURL() url.URL {...}
smyrman commented 3 months ago

Like for the enum and callback cases, the encapsulated field case above would ideally get passed a variable; in this case a *regexp.Regexp instance.

However, adding const would already help quite a bit, so if there are reasons that we can't do:

type StringRegexp[re var *regexp.Regexp] struct{}

Then:

type StringPattern[pattern const string] struct{}

would still go a long way.

ianlancetaylor commented 3 months ago

In the original post, I see

func (v Enum[L,T][lookup]) Value() (driver.Value, error) {

What is lookup here? I don't see it defined anywhere else. Is it a declaration? A type argument?

ianlancetaylor commented 3 months ago

OK, we see it now. The lookup is an argument to the type parameter list. And the instantiation has to pass in an argument for that argument. In this example it's passing in a map.

ianlancetaylor commented 3 months ago

What we see here is

func MustCreateLookup[T comparable](map[T]string) EnumLookup{
    // implementation doesn't matter.
})

type MyType = sqlenum.Enum[uint](enum.MustCreateLookup[MyType](map[MyType]string{
   MyTypeValue1: "value1",
   MyTypeValue2: "value2",
})

This appears to be actually calling a function at compile time. That can't work. What if the function panics? Or behaves differently in different calls. There is too much flexibility in Go to permit calling functions at compile time.

Or, perhaps it is being called at run time. But then why not just use an ordinary function? There is no obvious reason why a dynamically called function should be part of a type in some way. What advantage do we get from that?

Merovius commented 3 months ago

I'm not sure I really understand this proposal. Let me try to summarize it in my own words (a little bit stream-of-consciousness style, apologies) and tell me if I'm wrong.

The proposal is to add a second list of (let's call them) "value parameters" to type declarations. Instead of interfaces/constraints, the arguments would have a type and would be expected to be instantiated with a value. If a generic type is instantiated with two different values, they would constitute different type. That implies (as the compiler needs to be able to type-check), that either a) the values used to instantiate this second parameter list have to be constants and hence the types of value parameters must have underlying type one of the predeclared non-interfaces. Or b) that we'd introduce some way for the compiler to do inference on the possible values that a value argument can take, which would be dependent types, which Ian excluded from consideration. So value parameters would always be constants.

Now, what I'm struggling to understand is, what problems this solves and how. It seems the most interesting thing to talk about are the enum changes. The proposal text contains this:

package enum

// Lookup provides a two-way lookup
// between values of type T and strings.
type Lookup[T comparable] interface{
   Value(string) (value T, ok bool)
   Name(T) (name string, ok bool)
}

func MustCreateLookup[T comparable](map[T]string) EnumLookup{
    // implementation doesn't matter.
})

Now, the comment says "implementation doesn't matter". Am I understanding correctly, that the implementation would be something like this?

type EnumLookup[T comparable] struct {
    names map[T]string
    vals map[string]T
}

func (e EnumLookup[T]) Value(name string) (T, bool) {
    v, ok := e.vals[name]
    return v, ok
}

func (e EnumLookup[T]) Name(val T) (string, bool) {
    s, ok := e.names[val]
    return s, ok
}

// note the extra type parameter on `EnumLookup`, which the proposal text omits but seems necessary
func MustCreateLookup[T comparable](m map[T]string) EnumLookup[T] {
    names := maps.Clone(m)
    vals := make(map[string]T)
    for val, name := range names {
        if _, ok := vals[name]; ok {
            panic(fmt.Errorf("duplicate name %q", name))
        }
        vals[name] = val
    }
    return EnumLookup[T]{names, vals}
}

The proposal text then suggests a new library package enumsql, having the API (assuming the *MyType receiver was a mistake):

// note: we are not actually allowed to make the underlying type T. We need to make it struct{v T}.
// But let's ignore that for now.
type Enum[L enum.Lookup[T], T comparable][lookup L] T
func (v Enum[L,T][lookup]) Value() (driver.Value, error)
func (v Enum[L,T][lookup]) Scan(src any) error

Now, I'm not sure how this is supposed to work. As I mentioned above, ISTM for this proposal to meaningfully work, value arguments must always be constants and value parameter types must be basic types. So, already, the implementation I suggested above doesn't work - it is based on structs and maps and those aren't constants. Thus, we need a different implementation for the enum.Lookup[T] interface. So, at this point, I have to respectfully disagree with the proposal text saying "implementation does not matter". What exactly is EnumLookup supposed to be in the proposal text?

Now, if we can make all of this work, I'd compare it to this:

package enum
// as before (in the "no language change" section)
package enumsql

func Scan[T any](e enum.Lookup[T], p *T, v any) error {
    s, ok := v.(string)
    if !ok {
        return fmt.Errorf("unsupported type %T", v)
    }
    val, ok := e.Value(s)
    if !ok {
        return fmt.Errorf("invalid value %q", s)
    }
    *p = val
    return nil
}

func Value[T any](e enum.Lookup[T], p *T) (driver.Value, error) {
    s, ok := e.Name(*p)
    if !ok {
        return nil, fmt.Errorf("invalid value %v", *p)
    }
    return s, nil
}
package main

var myTypeLookup = enum.NewEnumLookup[MyType](map[MyType]string{
   MyTypeValue1: "value1",
   MyTypeValue2: "value2",
})

func (v MyType) Value() (driver.Value, error) {
    return enumsql.Value(myTypeLookup, &v)
}

func (v *MyType) Scan(src any) error {
    return enumsql.Scan(myTypeLookup, v, src)
}

So, ISTM that what this proposal saves is having to implement two trivial methods, that just delegate to a helper. That's not nothing, but it also doesn't seem a lot. An alternative way to solve this with today's language is to instead make the enumsql.Enum type something that can be embedded alongside MyType, which wouldn't require the boiler plate methods, but would change the representation of MyType. Yet a third way to solve it (which is how I tend to solve these things in practical applications right now) is to make the enumsql helper something along the lines of

type Value interface {
    sql.Scanner
    driver.Valuer
}

func Wrap[T any](e enum.Lookup[T], p *T) Value

This has the cost of requiring an additional Wrap call whenever sql.Rows.Scan or the like is called.

So I can see that there is some cost saved, with all these solutions. But ISTM the real utility here isn't so much to have "value parameters". The benefit does not really depend on T[]["Foo"] being distinguishable from T[]["Bar"], at compile time. But it's about being able to somehow associate additional data to a type, to "synthesize methods" as a helper (in a somewhat roundabout phrasing). To me, this suggest that a type-level solution as this proposal uses, is the wrong approach. I think the issue about the value parameters having to be constants (which essentially makes them useless for this use case) is just a symptom of that.

smyrman commented 3 months ago

What we see here is

func MustCreateLookup[T comparable](map[T]string) EnumLookup{
    // implementation doesn't matter.
})

type MyType = sqlenum.Enum[uint](enum.MustCreateLookup[MyType](map[MyType]string{
   MyTypeValue1: "value1",
   MyTypeValue2: "value2",
})

This appears to be actually calling a function at compile time. That can't work. What if the function panics? Or behaves differently in different calls. There is too much flexibility in Go to permit calling functions at compile time.

I think there is mistake in the issue description, and the syntax should be with square brackets:

type MyType = sqlenum.Enum[uint][enum.MustCreateLookup[MyType](map[MyType]string{
   MyTypeValue1: "value1",
   MyTypeValue2: "value2",
}]

Where the first list of square bracket parameter with this syntax are type parameters, and the second list are variables.

Which should be understood by the compiler as:

var lookup = enum.MustCreateLookup[MyType](map[MyType]string{
   MyTypeValue1: "value1",
   MyTypeValue2: "value2",
})
type MyType =  sqlenum.Enum[uint][lookup]

I would not expect the function to be run at compile-time, but at package initialization time (in this case). However, the return "variable" reference (type and address) must be resolved at compile time. Defining the variable inline is not essential to the proposal; it's just a way to make the variable scoped to the type declaration.

I think the relevant cases to do type declarations with inlined variable references are:

On type equality, types are only equal if they are initialized with the exact same variable reference.

Example of equal types:

var lookup = enum.MustCreateLookup[MyType](map[MyType]string{
   MyTypeValue1: "value1",
   MyTypeValue2: "value2",
})
// MyType1 and MyType2 are equal.
type MyType1 =  sqlenum.Enum[uint][lookup]
type MyType2 =  sqlenum.Enum[uint][lookup]

Examples of unequal types:

var lookup1 = enum.MustCreateLookup[MyType](map[MyType]string{
   MyTypeValue1: "value1",
   MyTypeValue2: "value2",
})
var lookup2 = enum.MustCreateLookup[MyType](map[MyType]string{
   MyTypeValue1: "value1",
   MyTypeValue2: "value2",
})

// MyType1 and MyType2 are NOT equal.
type MyType1 =  sqlenum.Enum[uint][lookup1]
type MyType2 =  sqlenum.Enum[uint][lookup2]

This also means that the following two types are not equal:

// MyType1 and MyType2 refer to different addresses and are _not_ equal.
type MyType1 =  sqlenum.Enum[uint][enum.MustCreateLookup[MyType](map[MyType]string{
   MyTypeValue1: "value1",
   MyTypeValue2: "value2",
})]
type MyType2 =  sqlenum.Enum[uint][enum.MustCreateLookup[MyType](map[MyType]string{
   MyTypeValue1: "value1",
   MyTypeValue2: "value2",
})]

Or, perhaps it is being called at run time. But then why not just use an ordinary function? There is no obvious reason why a dynamically called function should be part of a type in some way. What advantage do we get from that?

The advantage I see, comes when declaring field types for use with models. When you define a model, you typically don't want to initialize it with information of how each field should be parsed and/or validated; instead you want the zero-value to be self-descriptive, and you want each model field to contain enough information about how it should be treated, without initialization.

In other words, value parameters are trying to solve the same problem as struct tags, but for more cases. A struct-tag uses string-like syntax to tell libraries (that must rely on reflection) how to treat each field of the struct. This is additional (and constant) information that is embedded along with other information on the field, such as it's name and type. Value parameters aim to allow plain code (without reflection) to do exactly the same. By declaring a struct field as Enum[][lookup], you embed more complex information about how this field should be treated, without having to redeclare all the methods of the Enum type into a new field type.

Use-cases within model field definitions could be:

  1. Define fields in order to control conversion to/from different formats. E.g. API or DB models. Field validation without struct tags could also be part of this.
  2. This comment describe the use-case of encapsulated fields. Here the encoding/decoding is still relevant, but with the additional use-case of guaranteeing valid state.

For the record, I like the proposed syntax in this comment better. I am picturing a variant of that syntax that accepts var statements would look like this:

type Enum[L enum.Lookup[T], T comparable, lookup var L] T

But what I like with syntax described by @ianlancetaylor , is that it allows controlling scope. E.g. start with const only without closing the door on a future var.

As far as I can see, a const offers similar capabilities as var, except it becomes a programming exercise to resolve the const reference into a var.

Merovius commented 3 months ago

@smyrman I think it would be useful to phrase your proposal in terms that are established in the spec or at least widely used in the community (and, of course, operatively defined where you introduce them newly, like "value parameter"). Note that Go doesn't have the notion of a "reference". And that is not nitpicky: It has pointers, but in general, there is no way to tell at compile time whether two pointers are equal. And not everything can be made into a pointer (that is, not every expression is addressable). Being precise about these things would naturally ask (and hopefully answer) some of the questions that have come up in this discussion.

AIUI, you are suggesting in your last comment that

  1. The value argument to instantiate a value parameter must be an identifier referring to a package-scoped variable or
  2. If it is an identifier referring to something that is not a package-scoped variable…? Perhaps this should error? I don't know.
  3. If it is not an identifier, then T[][Expr] is interpreted as syntactic sugar for var _tmp = Expr /* at package scope */; T[][_tmp].

Did I understand that correctly?

I'll note that under those semantics, there is no way to refer to T[][Expr] elsewhere (making it a fundamentally useless type), unless it is given a new name via alias or type declaration. That would make these pretty singular, as using them as type-literals becomes extremely limited (there is very little you can do with var x T[][Expr]).

As far as I can see, a const offers similar capabilities as var, except it becomes a programming exercise to resolve the const reference into a var.

For the record, that is only the case if we restrict the possible expressions that can be used to instantiate a value parameter, roughly as I said above. General expressions can not be resolved at compile time.

smyrman commented 3 months ago

@Merovius, let me reply to you comments as well. Will start with the last one.

1 and 2 seams reasonable restrictions. 3, yes, this is how I picture it (except that in a final version, the _tmp should not be available for other code inside the package).

We could add more restrictions as well, like only allowing const parameters to be declared in type aliases, certainly for a proto-type. In general, I think a requirement should be that a proto-type can generate code that will compile and run today. Anything that isn't consistent with that, should be considered out of scope.

ll note that under those semantics, there is no way to refer to T[][Expr] elsewhere

Correct. You would have to use type aliases to be able to refer to concrete initialized types outside of the package (unless you expose your package variables). If the expression is a regex for instance, that makes a lot of sense; you need the type alias when dealing with such values. Or rely on the interface that is implemented by these types.


Finally, let me reply to your first comment. I agree to the limited value of the enum case. You understanding of how the EnumLookup would look like, is also correct. Also, the const work-arounds I was thinking of already works with types, now that I think about it more closely:

Assume:

package sqlenum

type Enum[T comparable, K any] T

func (e Enum[T,K]) Value() (driver.Value, error) {
    // Assume `enum.Lookup[T](k)` will check a registry to find an `EnumLookup` for type `T` with key `k`;
    // and either panic or return an empty lookup when not found. 
    var k K
    v, ok := enum.Lookup[T](k).Name(e)
    if !ok {
        return nil, fmt.Errorf("val")
    }
}

Then an sqlenum.Enum type could be initialized as:

type myKey struct{}

func init() {
    // Assume MustRegister with register a loookup for `myKey{}` in the `enum` package or panic.
    enum.MustRegister[MyType](myKey{}, enum.MustCreateLookup[MyType](map[MyType]string{
        MyTypeValue1: "value1",
        MyTypeValue2: "value2",
    }))
}

type MyType = sqlenum.Enum[uint, myKey]

}

This wouldn't become any better with const.

Edit: for the case of enums, you could even use the (zero-value) of the Enum type itself as a key.

package sqlenum

type Enum[T comparable] T

func (e Enum[T]) Value() (driver.Value, error) {
    // Assume `enum.LookupFor[T]` will check a registry to find an `EnumLookup` for type `T` with the
    // zero-value of `T` as a key.
    v, ok := enum.LookupFor[Enum[T]]().Name(e)
    if !ok {
        return nil, fmt.Errorf("val")
    }
}
func init() {
    // Assume MustRegister with register a loookup for `myKey{}` in the `enum` package or panic.
    enum.MustRegisterLookup[MyType](
        MyTypeValue1: "value1",
        MyTypeValue2: "value2",
    })
}

type MyType = sqlenum.Enum[uint]

The code for the enum package registry is omitted, but could be implemented by a var map[any]any with type-casting performed in the LookupFor accessor.

ianlancetaylor commented 2 months ago

This is a complex set of ideas and everybody seems to have trouble understanding it. The need for this is unclear. It's not clear what important problem this solves that could be worth this additional complexity.

For these reasons, and based on the above discussion, this is a likely decline. Leaving open for four weeks for final comments.

smyrman commented 2 months ago

Closing it sounds like the right choice. I will give some closing comments for my point of view. Thanks for good questions & comments.

The initial use-case proposed in the issue (enum translation), turned out to be a bad one. At least for us, we where able to come up with a good alternative implementation that wouldn't benefit from any language changes. Discussion of enums on the issue comes to a similar conclusion; placing this information on the type is probably a symptom of the design.

A place where I later thought the proposal to still have some merit to us, is for the encapsulated field use-case. The linked example shows an encapsulated regex field; we do have more encapsulated field cases where a const-only variant of the proposal would be useful (to us). But the big question is how much value this proposal would give to others.

I have no indication that encapsulated fields is a common pattern in Go. I think for this language feature to be considered, it would need multiple good use-cases with well-understood gains.

ianlancetaylor commented 1 month ago

No change in consensus.