Make an RFC to make bindings immutable by default or remove mutable bindings altogether

[97jaz]

In Racket, bindings are presumed to be mutable. (Within the binding's module, it is legal to set! the binding's value.)

Since most bindings are not mutated (and mutation of bindings is generally discouraged in Racket), this seems like the wrong default. Instead, let's either:

• require mutable bindings to be annotated as such; or
• not have mutable bindings at all and instead require the use of explicit boxes.

Regarding the first option, the specific syntax would depend on whatever syntax is chosen for racket2, but in the current syntax you could imagine something like:

(define-mutable x 0)
(let ([mutable x 0]) ...)

Regarding the second, a poster on racket-users recently said, in response to the same suggestion:

I'd very much like to avoid having to introduce extra references merely 
to make something mutable.  It costs.

I understand the concern, but as a practical matter, many compilers will convert mutable bindings to boxes anyhow. (Chez certainly does.) It's a common approach to implementing safe-for-space closures in the presence of mutation.

[jackfirth]

I strongly prefer the second option. It means you never have to guess whether a variable is being set! or not. Additionally, it means code that reflects over module bindings (such as tools I'm working on to extract contracts from values and check that they match their documented contracts) never have to concern themselves with bindings mutated underneath them.

[sorawee]

One concern: what if I want to create a language that allows set!? Currently the macroexpander has a special treatment for identifier macros with respect to set!. If I were to create my own my-set! in a world without set!, would I lose any expressive power, or make it significantly more difficult?

[jackfirth]

I'm not sure, but I suspect a custom language could implement set! by converting identifiers used mutably to use a box instead. As far as I can tell the macroexpander's special treatment of set! is primarily via the "set transformer" protocol. I think a proof-of-concept language that implements set! in terms of immutable bindings and boxes should be a prerequisite for this change.

[97jaz]

@jackfirth says:

It means you never have to guess whether a variable is being set! or not.

True, though I imagine in the first case, you'd do pretty well just by assuming that all variables explicitly annotated as mutable are set! at some point.

@sorawee says:

If I were to create my own my-set! in a world without set!, would I lose any expressive power, or make it significantly more difficult?

It would be more difficult, because you would need to perform assignment conversion yourself. That is, you'd need to:

• analyze the code to figure out which bindings are mutable (however that's determined in your language)
• transform mutable bindings into explicit boxes
• transform uses of my-set! into set-box!

The difficulty of the first part depends on how your language works. To reproduce Racket's current rules, you'd just need to look through the variable's own module to see if any code set!s it. If your language required explicit annotation (as in my first suggestion), you wouldn't even need to do that. If your language wanted to allow the variable to be set anywhere, even by code outside the module, you could do an expensive control-flow analysis or just preemptively turn all variables into boxes. (Not great for performance, that last one.)

In other words, my-set! is easy to implement right now, if you want to reuse a lot of Racket's current rules around set!.

[sorawee]

Yup, Rosette in fact does this "transform a mutated variable to a box" strategy, but it uses syntax-id-rules which transitively uses make-set!-transformer. If the language doesn't have set! in the beginning, I suspect it will be difficult to implement it.

See https://github.com/emina/rosette/blob/efe22c924b93e11ff607428b148901be9a739e57/rosette/base/form/module.rkt#L143

[tgbugs]

While the evils of mutability are widely known, if someone is prepared to confront them, why force them to do awkward and unergonomic things with the language as punishment? Can't the primary concern about set! be dealt with simply by overriding its definition to trigger an error whenever it is used? Why remove a widely used construct from the language that many others find useful when you can solve the problem of wondering whether a variable is being set! in your own code by causing any usage of set! to trigger an error? Forcing paradigms onto users of the language seems to run directly counter to the entire spirit of Racket2.

[97jaz]

Can't the primary concern about set! be dealt with simply by overriding its definition to trigger an error whenever it is used?

In a word, no:

1. I don't have any particular problem with set!. I have a problem with variables being mutable by default.
2. Let's take my first suggestion: that we explicitly annotate mutable variables. How does triggering an error on set! help here? If I annotate a variable as mutable, I obviously expect it to be set!, and an error would be unwelcome.
3. @jackfirth can correct me if I'm wrong, but I'm fairly confident that his concern with knowing whether bindings are mutated is one of static analysis. At any rate, I know that my own concerns are all about static analysis -- not just the compiler's either, but the human reader's. I want to know if something is mutable by looking at the binding site, rather than needing to look at all of the use sites.

Why remove a widely used construct from the language...

Is set! widely used? I would be surprised if most Racket programmers didn't treat it the way I do: generally avoiding it but using it on occasion. I say this as someone who has been using Racket (and PLT Scheme in its former incarnation) for nearly 20 years.

But, again: my first proposal doesn't call for the removal of set!. Would you be opposed to annotating mutable variables? (I think you'd need to use them a lot for that to be a burden, and maybe this really does point to a dramatic difference in style.)

[sorawee]

I strongly prefer the second option. It means you never have to guess whether a variable is being set! or not. Additionally, it means code that reflects over module bindings (such as tools I'm working on to extract contracts from values and check that they match their documented contracts) never have to concern themselves with bindings mutated underneath them.

I'm not sure if I follow. Is this from the point of human readers or static analyzers? Static analyzers can figure out which variable is mutated easily (see the code of Rosette I posted above). In fact, if you want to do static analysis, set! is far preferable than box because box is a value which can flow dynamically. set! on the other hand gives you first order information that can be analyzed statically.

[97jaz]

In fact, if you want to do static analysis, set! is far preferable than box because box is a value which can flow dynamically. set! on the other hand gives you first order information that can be analyzed statically.

That depends on what you're trying to learn from your analysis. Let's not move the goalposts here. We're talking about knowing statically whether a variable (not a data structure) is mutated. If you remove set! from the language (the option Jack prefers) that analysis takes zero time, because the a priori answer is no. If you explicitly annotate variables, then you know the answer from the binding site.

Both of these are simpler analyses then scanning the module for occurrences of set!.

[sorawee]

Ah I see, thanks for clarification.

Question about Option 1: what will happen if someone annotates a variable as mutable, but use it immutably? Is it considered an error?

I have arguments against both sides.

• Make it an error: four years ago, Go used to error when you have an unused import (I don't know how it is right now). This looks good in principle but in practice becomes very annoying. For instance, when I want to debug something, I need to put in both debug and its import statement. After debugging is done, I also need to remove both.

• Not make it an error: Pyret requires shadowing to be explicit via a qualifier shadow. As a TA in two classes that use Pyret, I saw A LOT of students write shadow even though they don't need to. I suspect that some students saw an error message that shadow is needed at some places, so they ended up writing shadow everywhere preemptively to silence the errors. Even I myself wrote unnecessary shadow sometimes due to how I refactored code and forgot that it's not needed to write shadow anymore. @jpolitz and @blerner might have anything more to say about this.

  Pyret in fact also requires a qualifier for mutable variables (var), but no student (at least for the classes I TA'ed) uses it because they are asked to not use mutable variables throughout the course.

  Another place to look at is JavaScript, which has const and let qualifier. It's really a disaster. I observed that people who are in PL area tend to use const, but most software engineers don't really care. Some even advocate for using let everywhere.

  I believe that without enforcement, this qualifier will end up being meaningless.

[97jaz]

Question about Option 1: what will happen if someone annotates a variable as mutable, but use it immutably? Is it considered an error?

I never considered that. My inclination is to say no. Let's say we were to make it an error if a mutable variable were never mutated in a program. First of all, that needs to be a conservative analysis with respect to things like eval. You can't, in principle, tell ahead of time if a program that uses eval will try to mutate a binding. Currently, Racket says: "If it isn't lexically apparent that a binding is mutated within its own module, we'll treat it as immutable." Which is a fine idea, but I don't think it works so well to say: "This variable that you marked as mutable -- we can't easily tell if it's going to be mutated by your program, so we're going to consider that an error."

In fact, it seems like Racket's current ad hoc restriction of variable mutation to the variable's home module should be lifted if we add explicit annotations.

I'm familiar with the const/let thing in JS. I suspect that the situation in Racket would be very different, because (as I noted earlier in this discussion) I don't think Racketeers are already in the habit of mutating variables all that often, whereas Javascripters certainly are. Your comments about the shadow annotation in Pyret are really interesting, though.

[sorawee]

In fact, it seems like Racket's current ad hoc restriction of variable mutation to the variable's home module should be lifted if we add explicit annotations.

CC: @gus-massa in case you have any thought about this.

[gus-massa]

It's a complex question. I like immutability, but it will increase the friction for users of other languages that are making a transition, and it will make it more difficult to port code from other languages. (My strategy is to just cut&paste the code, make a line by line translation, and when it is working racketify it.)

I think I prefer define-mutable, but not raise an error in case it is not mutated, and don't allow mutations from other modules. But I'm not convinced that this is the best combination.

Using syntax-id-rules it is possible to use a hidden box, and let the code modify the box, and export a mutable or a immutable fake reference. I guess it is necessary to have a common set! in the base language, in spite perhaps it is used only for expanding the syntax-id-rules to set-box! or something. I'm not sure what happens with the top-level anyway.

(Note: IIRC Chez Scheme replace the mutable variables x with a mutable (cons x #%undefined) because it is more efficient in Chez Scheme. Perhaps the optimizer can change the boxes that don't escape to cons?)

[jpolitz]

shadow is orthogonal to mutability from a semantics POV, and has much different consequences for program behavior than mutability, so it may not be the best point of comparison. Its main use case is to let a programmer re-use a good name when necessary, while having it by default disallowed to re-use a name in nested scope.

In Pyret, it's not an error to create a mutable variable with var and then never mutate it. This would make declaring a variable at the REPL basically unusable, since you couldn't declare and update it at separate entries, OR the static REPL checking would be different indeed from program checking. As much as possible, we've tried to keep REPL entries consistent with the rules for binding and static checking at the top-level of definitions. I would offer that this is probably a design point that matters a lot for Racket and I don't see mentioned here.

var and shadow in Pyret both have a useful property, which is that removing a var or shadow keyword either leaves the program's behavior unchanged (if the name was never mutated or shadowed), OR gives a static error that some name is now being used inconsistently, and will report the just-changed binding as one of the locations in the error message. Similarly, adding a var or shadow declaration to a binding in a program that didn't already have a static error won't change its behavior.

[mflatt]

527