Reference parameters

[lrhn]

Dart currently only allows passing arguments by value (where object references are values). However, the language has first class closures which can close over variables, so you can effectively pass a reference to a variable by passing a getter function and a setter function:

class Ref<T> {
  final T Function() get;
  final void Function(T) set;
  Ref(this._get, this._set)
  T get value => get();
  set value(T value) { set(value); }
}
...
int next(Ref<int> counter) => counter.value++;
..
   int x = 0;
   var ref = Ref(() => x, (int v) { x = v; });
   ...
      var list1 = [for (var i = 0; i < 100; i++) next(Ref)];
      var list2 = [for (var i = 0; i < 100; i++) next(Ref)];

It's not particularly convenient, but it does work.

So, Dart could introduce reference parameters without actually needing to extend the power of the language, just the expressibility.

Say:

int next(int& counter) => counter++;
// or 
int next(ref int counter) => counter++;

where int&/ref int is a reference to a mutable integer variable. A final int& counter/final ref int counter parameter would be immutable reference to a final or mutable variable. (Or we can even say in int counter for read-only, out int counter for write-only and inout int counter for either, if we dare reuse variance notation for variables.)

A reference parameter must be passed a suitable left hand side as argument. Either a variable (local, static, instance) or an index operator access — the things that can currently be assigned to. A final variable cannot be passed to a non-final reference parameter.

The local reference variable is then an alias for the left-hand side that was passed to it. Reading from the reference variable will read from the underlying variable. Writing to the reference variable will write to the underlying variable. A reference variable can be passed as value to another reference parameter. (We can perhaps even declare reference variables (aliases), like int& x = y;, and a reference variable must be initialized eagerly, because initialialization is different from assignment, which assigns to the underlying variable. Or it might just be too confusing, and we should just restrict it to parameters).

Function types will distinguish reference paramaters from non-reference parameters. Neither is a subtype of the other, so void Function(int) and void Function(int&) are unrelated types. Technically, we could probably allow void Function(int) to be a subtype of void Function(int&), but it means the dynamic calling semantics must check which one it is to figure out whether to pass the variable by reference or by its value. It's better to just distinguish the two.

Effectively, it is as if there are hidden class types Ref<T> and FinalRef<T> (or InRef/OutRef/InOutRef if we want full generality), and invocations of a function with a Ref<T> parameter will implicitly create the Ref instance for you, capturing the getter/setter functions into a structure like the one shown above. And that would be a possible implementation strategy, although I hope implementations can do better.

That makes references a type just like any other type, just a hidden type with special methods that can't be accessed directly, with implicit creation of the value where needed.

Usage

With reference parameters, we can introduce helper functions like:

void swap<T>(T& v1, T& v2) {
  T tmp = v1;
  v1 = v2;
  v2 = tmp;
}
/// Returns value of [value], then sets it to false.
///
/// When used with variable initialized to `var first = true;` will return `true` on the first
/// call, then `false` on all following calls.
bool firstOnly(bool& value) => value && !(value = false);

/// Returns the negation value of [value], then sets it to `false`.
///
/// When used with variable initialized to `var first = true;` will return `false` on the first
/// call, then `true` on all following calls.
bool notFirst(bool& value) => !value || (value = false);

/// Returns the negation of [value], and sets it to the new value.
///
/// Repeated callas with the same variable will return alternating values.
bool toggle(bool& value) => value = !value;

which allows code which reads better by abstracting over a sub-part of an algorithm, even if it affects local variables:

bool first = true;
for (var value in values) {
  if (notFirst(first)) accumulator.add(",");
  accumulator.add(value);
}

or

void shuffle([Random? random]) {
  random ??= Random();
  for (var i = 1; i < list.length; i++) {
    swap(list[i], list[random.nextInt(i + 1)]);
  }
}

This allows you to have user-written operations like the composite operations that are currently language-only, +=, ++, ??=, etc.

  // The ones we already have:
  int preIncrement(int& value) => value = value + 1;
  int postIncrement(int& value) { var tmp = value; value = value + 1; return tmp; }
  int add(int& ref, int value) { ref = ref + value; }
  // Alternatives that we do not have:
  void update<T extends Object>(T?& value, T ifNotNull(T value), T ifNull()) {
    var v = value;
    value = v == null ? ifNull() : ifNotNull(v);
  }
  void countDown(int& counter, void Function(int) action) {
    while (--counter >= 0) action(counter);
  }

(That is, we can get closer to user-defined control structures.)

Stretch goal

We can also allow extension methods on references:

extension _ on int& {
  int get next => ++this;
}
extension <T extends Object> on T?& {
  void update(T Function(T) ifNotNul, T Function() ifNull) {
     var value = this;
     this = (value == null) ? ifNull() : ifNotNull(value);
  }
}

That reads better than having to pass the reference being updated as a parameter.

Cons

The risk, as always, is that it becomes harder to read code.

If it's considered too "magical" that notFirst(first) can change the value of first, even though it looks like a normal function argument, we could also add an operator at the call-site, so it would be notFirst(&first). Then it's clear at the call-point too that something is going one. That doesn't work well for extensions on references. It can work, but &list[x].update(...) does not have a clear delimiter for which expression is the reference. Is it &list or &list[x]? Might need syntax like &(list[x]), or even ref(list[x]) which makes it more verbose. An alternative would be using &.foo() to call the extension method, but that doesn't work for operators. All in all, syntax is tricky for reference extensions.

If references are implicitly wrapped getter/setter functions, then they are objects, assignable to Object, but with no clear way to convert back. Would we allow as int& as a type cast? (Probably necessary, if assigning to Object is allowed, and not making that allowed is a significant change to the type system). Are references invariant? Should they be? (I guess final references are covariant, mutable references are invariant, and if we go for in/out/inout then out references are contravariant). In either case, that would require variance (#524) to exist in the language.

[ykmnkmi]

ref, similar to vala.

[ykmnkmi]

Vala Parameter Directions

[Levi-Lesches]

My two cents: Make the & required at the call site as well. It makes using a function that changed to pass-by-reference easier to spot and callers have to think about any consequences (which helps readability).

[munificent]

We would probably want both ref and out parameters, like in C#. Otherwise, the definite assignment analysis doesn't work as well because it can't tell which parameters will be definitely assigned by being passed by reference and which must be assigned before being passed as arguments.

This is definitely a thing we could do but, overall, I'm not convinced it carries its weight. I haven't found myself missing ref params much in Dart. The few times I do, it's usually just to have multiple return values, which could be more directly handled by support for tuples.

[lrhn]

But ... swap? :grin:

It's true that we have gotten this far without reference parameters, and that they probably have some risk of misuse, or uses that are just covering for the feature we really want, like multiple return values. Maybe even templates like #1353, where you want to abstract over a variable (although, in that case you probably want to abstract over the name and introduce new members with derived names).

I do think it could help with code-reuse in some places where we currently require you to rewrite the code (or introduce the get/set closures, which nobody does). Say

set foo(Foo value) {
  var current = _foo;
  if (!identical(value, current)) {
    _foo = value;
    _fooChanges.add(ChangeNotification(current, value));
  }
}

could become:

set foo(Foo value) => ChangeNotifier.updateWithNotification<Foo>(&_foo, value, _fooChanges);

everywhere you have that pattern, and depend on a single:

static void updateWithNotification<T>(T? target, T value, StreamController<ChangeNotification<T>> notifier) {
  var current = target;
  if (!identical(current, value)) {
    target = current;
    notifier.add(ChangeNotification<T>(current, value));
  }
}

So, the feature has expressive power, even if it can be desugared to existing Dart code, because it can make code-reuse shorter and practical (and the desugaring to closures can easily end up being more verbose than the original).

(And introducing the notion of a LHS-abstraction would make the specification easier, we can define += in term of the reference of its first operand, without having to re-specify it for every kind of variable and index operation. Basically, anywhere the spec requires an assignable expression.)

[munificent]

We've talked recently about statement-level or expression-level metaprogramming through something like inline functions or compile-time execution. I wonder if we could subsume this feature under that. If we had something like inline functions where the parameters became transparent references to the original arguments, it might be possible to cover these use cases.

[munificent]

the macro receives its parameters basically by name.

Macros receive the parameter expression as unevaluated syntax. You could write a swap() macro that does what you want, but the macro author would have to be careful around variable capture and hygiene. (Also, we don't currently have any plan for expression level macros.)

[Levi-Lesches]

at the very least there should be an hint on the call site.

Hence the recommendation to use & at the call-site, like notFirst(&first)

[lrhn]

That looks like

class Ref<T> {
  T? value;
  Ref([this.value]);
}

with more steps. If you want add a hasValue, you need to do more work, but what you show here is functionally equivalent to a nullable public variable. Change <T> to <T extends Object>, and you can use null to represent no value.

A more complete version could be:

final class Ref<T> {
  ({T value})? _value;
  Ref(T value) : _value = (value: value);
  Ref.empty() : _value = null;
  bool get hasValue => _value != null;
  T get value => (_value ?? (throw StateError("No value"))).value;
  set value(T value) { _value = (value: value); }
}

It can distinguish a value of null from no value, like the list based one, just with a simpler wrapper.