proposal: Go 2: immutable type qualifier

romshark commented 5 years ago

This issue describes a language feature proposal to Immutable Types. It targets the current Go 1.x (> 1.11) language specification and doesn't violate the Go 1 compatibility promise. It also describes an even better approach to immutability for a hypothetical, backward-incompatible Go 2 language specification.

The linked Design Document describes the entire proposal in full detail, including the current problems, the benefits, the proposed changes, code examples and the FAQ.

Updates

October 7th: This proposal is approaching its second revision addressing major flaws such as const-poisoning, verbosity, const-keyword overloading and others.

Introduction

Immutability is a technique used to prevent mutable shared state, which is a very common source of bugs, especially in concurrent environments, and can be achieved through the concept of immutable types.

Bugs caused by mutable shared state are not only hard to find and fix, but they're also hard to even identify. Such kind of problems can be avoided by systematically limiting the mutability of certain objects in the code. But a Go 1.x developer's current approach to immutability is manual copying, which lowers runtime performance, code readability, and safety. Copying-based immutability makes code verbose, imprecise and ambiguous because the intentions of the code author are never clear. Documentation can be rather misleading and doesn't solve the problems either.

Immutable Types in Go 1.x

Immutable types can help achieve this goal more elegantly improving the safety, readability, and expressiveness of the code. They're based on 5 fundamental rules:

I. Each and every type has an immutable counterpart.
II. Assignments to objects of an immutable type are illegal.
III. Calls to mutating methods (methods with a mutable receiver type) on objects of an immutable type are illegal.
IV. Mutable types can be cast to their immutable counterparts, but not the other way around.
V. Immutable interface methods must be implemented by a method with an immutable receiver type.

These rules can be enforced by making the compiler scan all objects of immutable types for illegal modification attempts, such as assignments and calls to mutating methods and fail the compilation. The compiler would also need to check, whether types correctly implement immutable interface methods.

To prevent breaking Go 1.x compatibility this document describes a backward-compatible approach to adding support for immutable types by overloading the const keyword (see here for more details) to act as an immutable type qualifier.

Immutable types can be used for:

Immutable Types in Go 2.x

Ideally, a safe programming language should enforce immutability by default where all types are immutable unless they're explicitly qualified as mutable because forgetting to make an object immutable is easier, than accidentally making it mutable. But this concept would require significant, backward-incompatible language changes breaking existing Go 1.x code. Thus such an approach to immutability would only be possible in a new backward-incompatible Go 2.x language specification.

Related Proposals

This proposal is somewhat related to:

20443
22876
22048

Detailed comparisons to other proposals are described in the design document, section 5..

Please feel free to file issues and pull requests, become a stargazer, contact me directly at roman.scharkov@gmail.com and join the conversation on Slack Gophers (@romshark), the international and the russian Telegram groups, as well as the original golangbridge, reddit and hackernews posts! Thank you!

gwd commented 5 years ago

Re "default immutable", would it make sense to add some sort of a keyword such that you can change the default for a specific file to being immutable? That would be backwards-compatible, but would allow people who want to use it an easy way to opt-in.

networkimprov commented 5 years ago

@gwd great idea! I filed an issue for that concept in the design doc repo:

https://github.com/romshark/Go-1-2-Proposal---Immutability/issues/23 - Mutability qualifier limited to package

romshark commented 5 years ago

@gwd the overloading of const proposed in the first revision of this document is now obsolete and we do indeed consider opt-in per-package solutions in the second revision that's currently a WiP.

I'm not yet sure whether the "everything is immutable by default in immutablity-aware packages" strategy is the way to Go because Go was and most likely will remain mutable by default. Immutability by default in immutability-aware packages might cause a lot of confusion (actually I'm not yet sure whether it will or not, but I tend to assume that it will).

The (probably) less confusing way would be to still keep everything mutable by default but make immutability-aware packages provide the mut and immut keywords which are not part of the older language specification. You then could enable those keywords using the //go:immutable compiler flag and when you want a function argument to be immutable for example you'd declare it as func PointerMatrix(a immut [][]*T).

When an old immutability-unaware package imports a newer immutability-aware package the immut and mut keywords are just ignored, but the compiler could still throw a warning when old code incorrectly uses newer code, like when you're mutating immutable global variables in immutability-unaware packages or similar.

EDIT: see the comment below

networkimprov commented 5 years ago

But you really ought to try a default-immutable proposal first to see if it flies :-)

romshark commented 5 years ago

@networkimprov imagine the following case (immutability is by default)

we have a new package b which has immutability enabled:

//go:immutable
package b

var C string = "immutable"

we have an old package a which is immutability-unaware:

package a

import "b"

func Func() {
  b.C = "new value" // dangerous!
}

Since a is not aware of immutability - the above code will compile, but might throw a warning like:

.a.go:6:6 WARNING: mutating variable of an immutable type immut string

The problem is that it's not obvious for a regular Go programmer, who's not yet familiar with the concept of immutable types, that b.C must not be mutated! b.C looks just like a regular mutable string variable. If the code was not immutable by default, like this:

//go:immutable
package b

var C immut string = "immutable"

...then it'd be less confiusing because you'd take the immut qualifier into account

However, I'm not 100% sure whether or not this is truely confusing, maybe it's not if there's a warning in both the linter and the compiler?

networkimprov commented 5 years ago

Well if you import a //go:immutable package, you're writing new code. If you prefer your code not be default-immutable, let keyword immut tag references to data from the immutable package. (immut would not apply to local-package data, nor within a default-immutable package.)

// NOTE: revised in a comment below

package a

import "b"

func f() {
  v := b.C          // OK, copied
  p immut := &b.C   // OK
  var x immut *T    // OK
  x = p             // OK

  b.C = T{}         // no
  *p = T{}          // no
  p = new(T)        // no
  x immut := T{}    // no
  x immut := new(T) // no
}

romshark commented 5 years ago

@networkimprov I just realized, that "immutability by default" might be dangerous! If we elude immut then older compilers will recognize b.C as a regular mutable string, which is not good at all! Chances are high you're gonna get yourself in trouble because the author of b assumed b.C will never be mutated. Mutating b.C might produce silent bugs which compile perfectly on older compilers.

It's better to make the immut keyword obligatory because older compilers will fail as immut is an unknown keyword from their point of view.

So, finally, it should rather look like this:

//go:immutable

// package b is taking advantage of the new immutability qualifiers
package b

type T struct {
  Name immut string
}

var C immut T = T{
  Name: "global",
}

// package a doesn't accept the "immut" and "mut" keywords
// but respects immutability of imported packages on newer compilers
package a

import "b"

func f() {
  /* local instance */
  l := b.T{}         // l is of type "mut b.T"
  l.Name = "foo"     // ERROR: illegal mutation of "immut string"

  /* copy */
  v := b.C           // v is of type "mut b.T"
  v.Name = "foo"     // ERROR: illegal mutation of "immut string"
  v = b.T{Name: "x"} // fine, v is mutable
  v.Name = "bar"     // ERROR: illegal mutation of "immut string"

  /* direct access */
  b.C = b.T{}        // ERROR: illegal mutation of "immut T"
  b.C.Name = ""      // ERROR: illegal mutation

  /* intermediate pointer */
  p := &b.C          // p is of type "mut * immut b.T"
  p.Name = "baz"     // ERROR: illegal mutation of "immut string"
  *p = b.T{}         // ERROR: illegal mutation of "immut T"
  p = new(T)         // fine, because the pointer is mutable ("mut * immut b.T")

  /* casting */
  var x *b.T = p     // ERROR: illegal casting "mut * immut b.T" to "mut *b.T"
}

compiling with an older compiler, which doesn't have a clue about //go:immutable: will fail
compiling with a newer compiler, which supports immutable packages: will fail due to illegal mutations and illegal casting

networkimprov commented 5 years ago

If //go:immutable doesn't stop a compiler that doesn't recognize it, then maybe

package b
immut

package b immut // alternatively

kirillDanshin commented 5 years ago

@networkimprov I like the oneline syntax, but if we scale it it can look bad:

package b immut nogc smthElse

romshark commented 5 years ago

@networkimprov

I personally think that a compiler flag like //go:immutable is okay for a language experiment while a proposal changing the package declaration syntax will almost certainly be rejected for obvious reasons (it's exceptional, it's kinda weird if other parameters get introduced, etc.).
Also, people are used to a Go where everything's mutable by default and will probably revolt when var C T from a newer package and var C T from an older one are different types even though they look exactly the same.

If we keep Go mutable by default and just introduce the immut and mut keywords through the //go:immutable immutability experiment, then we can drop the flag in the future, when immut and mut become an inherent part of the language specification. We won't however be able to drop the flag if we make those packages immutable by default because if we elude the flag later then old untouched code won't compile!

deanveloper commented 5 years ago

Comments should never be used to indicate language changes. Go uses comments to give hints to the compiler that you want something to be treated in a certain way, but they should never cause something to fail to compile. (They're also used with cgo, but that's a whole separate thing)

Either way, from a design standpoint, a comment should not affect the language itself in any way. If we do this it'd be much better with package b immut specifier or similar.

romshark commented 5 years ago

@kirillDanshin @deanveloper

package b immut nogc

looks kinda confusing, hard to read, especially if we consider adding more in the future. Argument-like parenthesis would look better IMHO:

package b (immut, nogc)

Also immut could be shortened down to mut for "mutability qualification".

networkimprov commented 5 years ago

Back to immutable-mutable package integration. I suggested immut to tag references in a mutable package to data from an immutable package; immut would not apply to local-package data, nor copies of immutable data.

@romshark then suggested immutable behavior in a default-mutable package, which I'd expect to cause the confusion he's concerned about, in https://github.com/golang/go/issues/27975#issuecomment-443225542.

I think the following rules are both consistent and safe:

package m             // default mutable

import "i"            // default immutable

func f() {
  var x * immut i.T   // OK
  p := &i.D           // OK
  x = p               // OK

  v := i.D            // OK, copied
  v = i.T{}           // OK
  v.a = 1             // OK

  *p = i.T{}          // no
  p = new(i.T)        // no
  i.D = i.T{}         // no
  i.D.a = 1           // no
  var x immut i.T     // no
}

networkimprov commented 5 years ago

And we really ought to seek a blessing from the Go Gods to continue this discussion, because we might be wasting our fingerwork :-)

@ianlancetaylor @robpike @griesemer @rsc, any thoughts on package-specific immutability?

More above, and here: https://github.com/romshark/Go-1-2-Proposal---Immutability/issues/23

romshark commented 5 years ago

@networkimprov

The laws of immutability must be respected even in the scope of regular immutability-unaware packages, otherwise immutability qualification doesn't solve any problem but creates a whole bunch of new ones when seemingly immutable stuff gets mutated silently from places you'd never expect.

And here's why:

A copy isn't always a deep copy because of pointer aliasing.

package i (immut)

type T {
  S string
  F immut *T
}

func (r *T) PotentiallyMutate() {}

func (r immut *T) ReadOnly() {}

var G T = T{
  F: &T{},
}

package main

import "i"

func main() {
  g := i.G    // Shallow copy
  g.F = nil   // ERR: illegal assignment
  g.F.F = nil // ERR: illegal assignment
}

i.G.F is only shallowly copied, thus mutating g.F.F will mutate G causing all sorts of trouble, because nobody ever expected G to be mutated from the outside! Making a deep-copy of i.G.F is also not an option for 2 reasons:

we'll decouple the two. Any state changes like a mutation in i.G.F.S will not reflect on g
it's too expensive for recursive structures, especially if we do it over and over again at runtime

Even when we create our own independent instance in the scope of an immutability-unaware package we still must obey the rules:

package main

import "i"

func main() {
  m := i.T{F: &i.T{}}
  m.ReadOnly()          // OK
  m.PotentiallyMutate() // OK, because m is mutable

  m.F = nil               // ERR: illegal assignment
  m.F.PotentiallyMutate() // ERR: mutating method on immutable m.F
  m.F.ReadOnly()          // OK
}

Why? because we didn't make i.T.F immutable for no reason! There's a reason why we did it and we cannot allow main to illegaly abuse i.T in a way the author of i never intended. Also implementing the "when something originates from an immutable package then obey the rules, otherwise don't"-logic would unnecessarily complicate the compiler.

What must be illegal is: using the immut keyword in a non-immut-package. The immut and mut keywords are only available in immutability-aware packages for backward-compatibility reasons.

var x * immut i.T

The above snippet should not compile in package main. It could only compile in package main (immut)

It'd sure be great to get feedback from actual Go-team members, but I'm yet to publish the second revision of the design document covering already mentioned criticism:

const poisoning (immutability genericity),
verbosity (MQP - mutability qualification propagation),
const overloading (per-package immutability)
immutability by default?

As soon as I've covered these issues I'll document them, publish the second revision and ask the Go-team for further feedback. I just don't like wasting anyone's time with ill-conceived ideas and immutability isn't on their hot-fix list as it seems.

ianlancetaylor commented 5 years ago

@networkimprov Personally I'm not a fan of having the language change on a per-package basis. That's typically the wrong granularity.

romshark commented 5 years ago

@ianlancetaylor currently I know of only 3 possible ways:

introduce the mut and immut keywords, potentially breaking any code that uses these names for other symbols keeping Go mutable by default.
- pro: easiest
- con: not so great for backward-compatibility, some older code might break due to name collisions
- con: no experimental phase

introduce the new keywords on a per-package basis with a new package-arguments syntax: package p (immut)
- con: requires a new package declaration syntax
- con: no experimental phase
- con: cannot be canceled as easily as 3.

introduce the new keywords on a per-package basis with a temporary //go:immut compiler-flag and call it the "Go immutability experiment". Remove the flag in the future and get to the same situation as 1. but give people time to migrate
- pro: experimental feature with potential to become part of the language
- con: introduces a new kind of compiler flag which rules the package scope and must be present in each file of the package

If we want immutability by default then 2. and 3. are the only options (except that we can't remove the flag over time, because the code from older Go packages and the newer ones - where everything's immutable by default - is naturally incompatible)
the check keyword proposal takes the 1. approach if I'm not mistaking, so it wouldn't be the only language feature to get introduced this way

ianlancetaylor commented 5 years ago

We can add new keywords in new language versions where needed. See the discussion in #28221.

beoran commented 5 years ago

Immutability can today already be achieved by using structs with purely value based semantics, avoiding pointers altogether. For example:

package main

import (
    "fmt"
)

type CannotMutate struct {
    i int
    s string
}

func NewCannotMutate(i int, s string) CannotMutate {
    return CannotMutate{i, s}
}

func DoesNotMutate(data CannotMutate) CannotMutate {
    data.i++
    return data
}

func (cnm CannotMutate) String() string {
    return fmt.Sprintf("CannotMutate: %d %s", cnm.i, cnm.s)
}

var hiddenState CannotMutate = CannotMutate{42, "Can't touch this"}

func State() CannotMutate {
    return hiddenState
}

func main() {
    data1 := NewCannotMutate(7, "hello")
    fmt.Printf("Data 1: %s\n", data1.String())
    data2 := DoesNotMutate(data1)
    fmt.Printf("Data 1: %s\n", data1.String())
    fmt.Printf("Data 2: %s\n", data2.String())
    data3 := State();
    fmt.Printf("Data 3: %s\n", data3.String())
    fmt.Printf("hiddenState: %s\n", hiddenState.String())
    data4 := DoesNotMutate(data3)
    fmt.Printf("Data 3: %s\n", data3.String())
    fmt.Printf("Data 4: %s\n", data4.String())
    fmt.Printf("hiddenState: %s\n", hiddenState.String())   
}

Try it here: (https://play.golang.org/p/BJiDNdxk9N_t) As long as you stick to this style, external packages cannot mutate your package's state, nor can they mutate your data's contents.

An immut keyword brings a lot of trouble without much additional benefit beyond sticking to a value base API as I show above. Furthermore, seeing the new rules for Go language change proposals (https://blog.golang.org/go2-here-we-come), immutability by default is definitely out of the question, since it would require a full rewrite of almost /all/ existing go code.

Such immutable API is practical and has been used in the wild frequently, see this github search: https://github.com/search?q=immutable+go

romshark commented 5 years ago

@beoran I'm sorry, but have you even read the problems section? Please read it, it's worth your time :) There were also a lot of discussions you should take into account.

If you still don't want to take your time reading then consider this: how do you want to safely handle slices then? Just hope the code does what the comments say? Copy the hell out of every slice? This proposal is about compile-time guarantees and clear definitions of APIs, again, please read it!

Immutability-by-default was one of the possible ways, and no, it would not require rewriting any existing code if it's enabled on a per-package basis! I now move away from it though for backward-compatibility and historical reasons. We don't want per-package immutability as it turns out, so we can't use immut-by-def. Also, Go1 is mutable by default, making Go2 the opposite would generate a lot of confusion. Go2 will probably have to remain mutable by default and just provide the mut and immut keywords.

beoran commented 5 years ago

Yes I have read that section, actually. :) But I think the problem lies not with the lack of an immut or const keyword. Rather, the basic problem is bad API design.

Too many go language APIs in the wild require a pointer to be passed where a struct is sufficient. Or a package variable is made directly accessible without a wrapper function. If we don't want mutability, then I think we should enforce it through an immutable API.

As for the performance of using structs in stead of pointers, this is an implementation detail, I think the current go compilers already can optimize this somewhat, and we should work on improving the optimization. In C, with gcc, when you pass and return structs, this is optimized and no copies are done. Go compilers can also learn how do this if they don't already.

As for slices, wrap them in structs with am immutalbe API, or why not, use arrays, and the problem is lessened considerably. In the link I provided there are examples of go immutale list libraries that help solve this problem.

I agree that immutability by default is off the table. But, the more I follow this topic, the less I am convinced that an immut keyword is the right way to solve the problems you mentioned.

I think the better solution is to avoid reference types (pointers, slices) as much as possible and use value types in stead.

romshark commented 5 years ago

@beoran pointers and copies are not interchangeable, there's a reason we have them and there's sometimes a reason to use them in APIs. But it's not only public APIs, it's also package-internal code that often needs to be protected, consider the following example:

Not just public APIs, it's compiler-enforced documentation & clear intentions

package a

type Object struct {
  // uniqueID must remain immutable once the Object is created
  uniqueID string
}

func NewObject() Object {
  return Object{
    uniqueID: generateUniqueID(),
  }
}

/* lots of code here */

How do you make sure Object.uniqueID is never changed after the Object instance is created? You'll have to make sure you never allow something like obj.uniqueID = "foo" to ever happen in the scope of your package, which might be big & open source, and a lot of people could be working on it pushing their pull requests.

Instead of relying on smart humans (which is an anti-pattern in software engineering), I propose this:

type Object struct {
  // Save to be exported
  UniqueID immut string
}

You just can't make this mistake, ever. You'll never get faulty pull requests because it won't compile if anyone messed it up.

Slices

// Find guarantees to never write to a
func Find(a []string, s string) int {
  return second(a, s)
}

// somewhere in another package, promises not to change b,
// because third promised it as well (sometimes in the past)
func second(b []string, _ string) int {
  return third(b)
}

// somewhere in a third package
func third(a []string) {
  a[0] = "whoops" // third(a []string) never guaranteed to never change a
}

How do you make sure a is never changed by Find? And by "never" I really mean never, not even over time and many many iterations. Writing a comment won't help, a comment is not enforced, it's a pure claim but code might change over time (bugs might get introduced). If we assume those functions are maintained by 3 different people then chances of silently introduced bugs become very high!

Do you really want to write wrappers & interfaces for each and every slice everywhere? Sounds like tons of boilerplate and trouble which could've been avoided with immut without a single line of additional code:

func third(a immut []string)

There's no way the maintainer of third can break his promise
There's no way the maintainer of second can introduce a silent bug because third decided to change its behavior over time because a change from a immut []string to a []string is a breaking one.

Copies, copies, copies, boilerplate, boilerplate, boilerplate

Consider the following example:

type T struct {
  internal []uint64
}

// Get returns a reference to the internal slice
func (t *T) Get() []uint64 {
  return t.internal // shallow copy return
}

We can't do that, we want to avoid T.internal from being randomly mutated from the outside, it's unexported for a reason, so we'll have to copy it:

// GetCopy returns a copy to the internal slice
func (t *T) GetCopy() []uint64 {
  cp := make([]uint64, len(t.internal))
  copy(cp, t.internal)
  return cp
}

But copying is expensive, how can we optimize our API? Well.. we could implement manual iterating:

// Len returns the length of the internal slice
func (t *T) Len() int {
    return len(t.internal)
}

// At returns an item from the internal slice given its index
func (t *T) At(index int) uint64 {
    return t.internal[index]
}

Let's look at the benchmark results

goos: linux
goarch: amd64
pkg: test
BenchmarkGet-12          1000000          1326 ns/op
BenchmarkGetCopy-12       300000          5610 ns/op
BenchmarkIndex-12        1000000          2227 ns/op
PASS
ok      test    5.338s

https://play.golang.org/p/NNcnq8DhWJV

The iteration interface is certainly much better than copying, but it's still not as fast as it could be, so... should we make internal exported and rely on our users to never mutate it? It's dangerous, but it's fast, it's a dilema!

A dilemma that could've been solved with:

func (t *T) Get() immut []uint64 {
  return t.internal
}

Still want your boilerplate?

I could go on and on... and actually I've already covered all this (and more) in countless discussions (I hope I did). It's all about software engineering, which is what programming becomes when you add time and other programmers.

beoran commented 5 years ago

I think your first example is an example of a bad API design. If it is critical that Object is immutable, then it should be implemented in its own package 'a/object'. This has the additional benefit that it makes testing and code review easier.

As for the slice example, a slice is in essence a pointer value, and so it should not be used at all if immutability is desired. The use of a slice is a hasty optimization here.

Yes, to do immutability in Go now, we need to write a lot of boilerplate. That's a common complaint against go, look at how we don't have enums or other similar conveniences. Generics will probably solve this problem once they get implemented. But for now, go generate, or immutable data structure libraries are feasible and sufficient.

I can agree that as the compiler stands now, a value based API is less performant than a pointer based one. I consider this a compiler and runtime optimization problem. Many functional language compilers only have pure value based semantics and optimize this splendidly. Go should also do that, but that is a different issue.

As I see it the main benefit of the immut keyword would be performance. The immut keyword would tell the compiler to pass data by pointer or slice, but to also give the passed argument value semantics. While I see the appeal of that, I feel it doesn't weigh up against the added complexity of the language.

Particularly, the function argument and result covariance problem leads to having to either having write everything function twice, once with mutable and once with immutable arguments and results, or having to do mutable/immutable casts all over the place, much like with const in C.

I think I can see where you are coming from, but my experience with immutability is different from yours. I think we will have to agree to disagree, and let the go authors weigh the arguments and decide.

rosun82 commented 5 years ago

Nice document; clearly you spent a while on it. I only briefly glanced over it.

Copies are the only way to achieve immutability in Go 1.x, but copies inevitably degrade runtime performance. This dilemma encourages Go 1.x developers to either write unsafe mutable APIs when targeting optimal runtime performance or safe but slow and copy-code bloated ones.

Not exactly the only way. An alternative approach is to have an opaque type with only exported methods that provide read-only access, which is how reflect.Type achieves immutability. The v2 protobuf reflection API also takes this approach. It has other downsides (like needing to manually create methods for each read-only operation), but pointing out that there are non-copy approaches to immutability.

Well, I guess this does not quite work for nested protos

vincent-163 commented 5 years ago

I would like to point out one more use of immutable type: as map keys.

Among all the comparable types available in go, there are only two variable-length types: interface{} and string, where string is essentially an immutable []byte. The only way to use variable-length slices of arbitrary value type as a map key is to convert them into string. Immutable types allow us to replace the byte in string with arbitrary value types and use pointers instead of value types in map keys.

dsnet commented 5 years ago

I would like to point out one more use of immutable type: as map keys.

While this proposal uses the term "immutable", it is more accurately describing a "read-only" view of formerly mutable values.

On the other hand, map keys need true immutability, which is a more restrictive model. Not only do we need the map key to be "read-only", but we also need to guarantee globally that there are no mutable views of that value (lest the value changes due to remote side-effects, causing all sorts of buggy map behavior).

zigo101 commented 5 years ago

Currently, slice/map/function values can't be used as map keys is not because they are not immutable, it is just because different people have different views on how these values should be compared. To avoid the confusions caused by different views, Go forbids comparing them.

JordanMcCulloch commented 4 years ago

Just wanted to say that this would be an amazing addition to Golang, and mention that Microsoft is in the races with their new open-source Project Verona language focused on memory safety (and is highly influenced by RUST).

Robula commented 4 years ago

I definitely would prefer immutable by default as Golang is function first and it's difficult to enforce developers to declare runtime constants as readonly or immutible. I personally really like how Rust does it, let mut ....

pbarker commented 4 years ago

I would be curious to see a meta-analysis on immutable defaults with regard to the following topics:

Would declaring mut in every mutable variable declaration cause more code bloat than accessors across current popular repos?
Would introducing mut cause a cognitive burden on understanding the language?

romshark commented 4 years ago

@pbarker That's how the Go compiler would act if:

the Go type system was immutable by default
Go would allow safe implicit casting (safe only)
Go would allow mixed-mutability types

https://gist.github.com/romshark/5d4650d837c1d87ef237e68ca1408280

IMHO we'd get easier debugging and reading by trading off:

language simplicity (e.g. most people will struggle to understand the difference between mut *T, * mut T and mut * mut T, etc.)
the ease of writing
and probably the ease of refactoring too (making code more reliable usually also makes it less flexible)

ORESoftware commented 4 years ago

how about const? oh wait 🤣

something that works with := is probably necessary, so maybe:

const a, var b, const c  :=  get3Results();

🤣

HONESTLY, I would say something like OCaml would be nice, like a quote after the variable:

a',  b,  c'  :=  get3Results();

if the variable has ' after it, it's immutable...it's also important to do this in function parameters (JS and TypeScript do not have this to my knowledge):

func HasImmutableParams(a string, b' bool, c int){
   // b is immutable
}

romshark commented 4 years ago

@ORESoftware

how about const?

We already came to the conclusion that the const keyword shouldn't be overloaded.

I would say something like OCaml would be nice, like a quote after the variable:
a',  b,  c'  :=  get3Results();  
if the variable has ' after it, it's immutable...it's also important to do this in function parameters

This proposal is about read-only types, not about immutable variables. I already explained why I find the type-based approach better than the variable-based one.

Splizard commented 3 years ago

I don't like the idea of immut and mut keywords.

Why not use the existing 'readonly' type-syntax that already applies to channels? (<-)

IE.

//You can assign read/write values to readonly types.
//(like with channels)
var readonly <-int = 5
readonly = 2

//However, if the type is a pointer type or struct, slice or map
//then it is a compiler error to mutate/write the underlying value.
var slice  <-[]int = []int{1, 2, 3}
fmt.Println(slice) //Allowed
slice[0] = 2  //COMPILE ERROR

var pointer <-*int = new(int)
pointer = new(int) //allowed
*pointer = 3 //COMPILE ERROR

type Something struct { Value <-*int }
var thing Something
thing.Value = new(int) //allowed
*thing.Value = 3 //COMPILE ERROR

var readOnlyThing <-Something

//allowed
readOnlyThing = Something{
    Value: new(int),
}

readOnlyThing.Value = new(int) //COMPILE ERROR

romshark commented 3 years ago

@Splizard what if you want an immutable slice of pointers to mutable objects though? IMHO, <- syntax looks very confusing:

/* Immutable slice of pointers to mutable objects */
<-[]* T<-
immut []* mut T

/* Mutable pointer to an immutable object */
*<- <-T
mut * immut T

* <-T
* immut T

Splizard commented 3 years ago

@romshark <- indicates that the semantic region of memory for a value is read only.

For a slice, this is the elements of that slice. For a pointer, this is the value being pointed to. For a map this is the values inside that map. For a struct, this is the fields of that struct. Etc

However, any pointers inside immutable types are still ordinary typed pointers. <- isn't recursive and needs to be added to any type with pointer semantics in order for the underlying value of that type to be made immutable.

//ie this is allowed
var slice = <-[]*int{new(int)}
*slice[0] = 3
pointer := slice[0]
*pointer = 5

//but this is not allowed
slice[0] = new(int) //COMPILE ERROR

//in order to prevent the underlying value of
//the pointers from being 
//modified then you need to create a
//read only slice with read only int pointers.
var safeslice = <-[]<-*int{new(int)}
*safeslice[0] = 3 //COMPILE ERROR

romshark commented 3 years ago

@Splizard we're going to run into issues because of the existing read-only and write-only channel declaration syntax.

According to this proposal, the following statement declares an immutable channel:

c := make(immut chan int, 1)
var r immut <-chan int = c
var w immut chan<- int = c

c = nil // Illegal assignment on read-only type
r = nil // Illegal assignment on read-only type
w = nil // Illegal assignment on read-only type

There's an obvious semantic conflict in case of the <-chan and chan<- syntax since var c <-chan int would declare a mutable variable. Alternative syntax like var c <-<-chan int would be very confusing and far from ideal IMHO. Specifying mutability on the variable like this: var <-c <-chan int is not supported by this proposal since I propose to define mutability on the data types.

To avoid overly verbose declarations such as var s immut chan immut * immut T, this issue suggest propagating mutability qualification in the type definition: var s immut chan *T.

Splizard commented 3 years ago

@romshark I'm confused.

This proposal is about read-only types, not about immutable variables. I already explained why I find the type-based approach better than the variable-based one.

Why is assigning nil to a variable with type immut chan int illegal?

For example, strings are immutable in Go and this is completely valid:

c := string("hello")
c = ""

I mean a string is almost an alias of <-[]byte.

I don't see a semantic conflict with channels, they already have a read-only 'immutable' type equivalent. <-<- chan int is the same type as <-chan int because the type is already read only. You don't have to like this notation but I don't see why there should be multiple ways in Go to declare a type to be read-only/immutable.

deanveloper commented 3 years ago

@Splizard While I like your proposed syntax for its consistency, it produces an ambiguity:

var chanOfSlices chan<-[]Type = nil

Is this (chan<-)([]Type) or chan (<-[]Type)?

romshark commented 3 years ago

@Splizard

Why is assigning nil to a variable with type immut chan int illegal?

Take a look at this example:

type S struct {
  ID immut int
  Name string
}

s := S{42, "foo"}
s.ID = 43        // no!
s.Name = "bar"   // fine
s = S{43, "bar"} // fine

var s2 immut S = S{42, "foo"}
s2.ID = 43        // no!
s2.Name = "bar"   // no!
s2 = S{43, "bar"} // no!

immut makes s2 read-only.

For example, strings are immutable in Go and this is completely valid. I mean a string is almost an alias of <-[]byte.

I initially picked the wrong title for this proposal. We're rather talking about read-only, not immutable types. A variable or field of type string is still assignable/writable, the underlying data, however, is read-only. const string is somewhat closer to immut string with the only difference being that contstants must be known at compile-time while immut types are just read-only once they're initialized.

Splizard commented 3 years ago

@deanveloper No, this ambiguity already exists in Go, try creating a channel of read-only channels. You need to use parentheses. chan (<-chan int) vs chan <-chan int

@romshark I suppose I should create a new proposal, as I reject the idea that a type can restrict a variable from being reassigned. Variables are reassignable by definition. What s2 = S{43, "bar"} // no! shows, is that immut changes the meaning of the variable and it is no longer a variable, it is a const pointer to a read-only type. IE const s2 <-*S = &S{43, "bar"}

romshark commented 3 years ago

@Splizard If immut wouldn't apply to variables then we'd lose a crucial feature of this proposal because a very common use case would be package-level read-only variables:

package something

var ErrSomethingWentWrong immut error = errors.New("something went wrong")
var Options = immut []string {"foo", "bar", "baz"}
var Dict = immut map[string]string {"foo": "bar", "baz": "faz"}
var DefaultLogger immut *Log = &Log{os.Stdout}
// etc.

Variables are reassignable by definition

No, not necessarily. Functional programming languages usually don't even feature reassignment. A variable is a placeholder for values unknown at compile-time.

it is a const pointer to a read-only type,

A constant's value must be known at compile time. A variable of immut type isn't required to be known at compile-time.

jfcg commented 3 years ago

Hi, I am in favor of adding only *const T which is a pointer type that can only read target data, no writing. This does not mean target data is constant, just that target cannot be changed by read-only pointer. No immutability for regular data (including pointers). Declarations are like

var x int
p := &x       // regular pointer
r := &const x // read-only pointer

var s *const int
s = &x // ok

p = r // forbidden without a cast
r = p // ok
// they should have different reflection kinds. design document does not talk about reflection.

*p++
*r++ // forbidden

var z struct {
    A int
    B string
}
r := &const z
r.B += "abc" // forbidden

func (r *const Object) Some() { // read-only method of type

Thanks..

romshark commented 3 years ago

@jfcg 3 issues:

How do you define read-only package-scope variables?
How do you define read-only struct fields?
How do you make maps and slices read-only?

If those issues remain unanswered then this should be a different proposal because it serves a different purpose.

jfcg commented 3 years ago

* How do you define read-only package-scope variables?
* How do you define read-only struct fields?
expose them with read-only pointers
* How do you make maps and slices read-only?
similarly for slices, []const T could mean read-only slice, cannot be used to write to underlying data. for maps, maybe map[K]const T could mean the same thing. In both cases expansion/deletion/overwrite could be forbidden.

Or, slice expansion could be allowed if there is capacity. For map, map[const K]const T could mean not-expandbale as well.

romshark commented 3 years ago

@jfcg If, and only if *const int is similar to const int in the sense that it cannot be mutated and assigned to then it'd be an interesting alternative to consider.

However, I see a potential problem as it'll lead to pointer abuse and cause increased number of allocations and hence generate more pressure on the garbage collector since objects referred to by a pointer escaping the function scope will be allocated on the heap. This contradicts one of the goals of this proposal: improve performance by avoiding unnecessary copying.

Also, it'd be impossible to have mixed-mutability types, which, I fear might lead to problems.

jfcg commented 3 years ago

@jfcg If, and only if *const int is similar to const int in the sense that it cannot be mutated and assigned to then it'd be an interesting alternative to consider.

However, I see a potential problem as it'll lead to pointer abuse and cause increased number of allocations and hence generate more pressure on the garbage collector since objects referred to by a pointer escaping the function scope will be allocated on the heap. This contradicts one of the goals of this proposal: improve performance by avoiding unnecessary copying.

Also, it'd be impossible to have mixed-mutability types, which, I fear this might lead to problems.

I think an example is better:

package mypkg

type ConfVar struct {
    A int
    B string
}

var pkgVar ConfVar

func PkgVar() *const ConfVar {
    return &const pkgVar
}

So, there is no global read-only pointer to mess with and no allocation. mypkg can modify pkgVar and outsiders have fast & read-only access to that package-level variable by one call to PkgVar(). You can write similar methods for allowing read-only access to private struct fields. Can you elaborate what you mean with examples?

romshark commented 3 years ago

@jfcg let's say you want a factory that returns objects with an immutable field:

type Object struct { ID *const uint64 }

type Factory struct { counter uint64 }

func (f *Factory) NewObject() *Object {
  f.counter++
  id := f.counter                // Will be allocated
  return &Object {ID: &const id} // Pointer escapes function scope
}

This approach leads to a performance penalty.

The read-only type approach doesn't:

type Object struct { ID immut uint64 }

type Factory struct { counter uint64 }

func (f *Factory) NewObject() *Object {
  f.counter++
  return &Object {ID: f.counter} // No pointers, no allocations
}

This is an oversimplified example. I know that we could just return *const instead, but we might want the object to be mutable with only the Object.ID field being immutable.

package mypkg

type ConfVar struct {
   A int
   B string
}

var pkgVar ConfVar

func PkgVar() *const ConfVar {
   return &const pkgVar
}

Second issue: pkgVar remains mutable within the package, which might not be desirable.

package mypkg

type ConfVar struct {
  A int
  B string
}

var PkgVar = immut ConfVar{
  A: 42,
  B: "foo",
}

Read-only types would guaranteemypkg.PkgVar to be immutable since it cannot be written to from both within and from outside of the package.

jfcg commented 3 years ago

type Object struct { ID immut uint64 }

type Factory struct { counter uint64 }

func (f *Factory) NewObject() *Object {
  f.counter++
  return &Object {ID: f.counter} // No pointers, no allocations
}
This is an oversimplified example. I know that we could just return *const instead, but we might want the object to be mutable with only the Object.ID field being immutable.

This case (or similar ones) does not even need read-only pointers, let alone full blown immutability:

type Object struct {
    id uint64
    Data string
}
func (o *Object) ID() uint64 {
    return o.id
}

Methods are perfectly fine and even they can be optimized out by compilers.

Second issue: pkgVar remains mutable within the package, which might not be desirable.

It is actually by design (for example). Only the package can update its confguration variables, sounds pretty straightforward to me.

package mypkg

type ConfVar struct {
  A int
  B string
}

var PkgVar = immut ConfVar{
  A: 42,
  B: "foo",
}
Read-only types would guaranteemypkg.PkgVar to be immutable since it cannot be written to from both within and from outside of the package.

What advantages do always-immutable variables have over typed constants?

Can you think of a real example where full blown immutability delivers undisputed advantage that struct methods, functions and read-only pointers cannot?

astrolemonade commented 3 years ago

Was this issue addressed by someone from the core language team? I noticed that this is a 3 years old issue and in some way I would like to know if it is worth it to keep me hyped for constant custom datatypes. Having something readonly sometimes makes a huge difference in programming and also helps you at debugging very much.

golang / go