What does an undecorated class type mean for nilability?

[mppf]

This issue is related to:

• nilable types #12614
• Specifying arguments with class type and any nilability? #13088
• What does an undecorated class type C mean? #12917

Suppose that we have class MyClass. What does un-decorated MyClass mean for nilability when used as a variable type, as an actual argument to a function accepting a type, and as a formal argument type?

Generally speaking the nilability proposal in #12614 follows along with the initial design for borrowed/shared/owned/unmanaged, in which MyClass means borrowed MyClass. In particular the nilability proposal in #12614 expects that MyClass means a non-nilable class pointer but something of type MyClass? can be nil.

However, there are two related issues that suggest that we should consider possibly changing this interpretation:

• Over on #12917, we are thinking about interpreting MyClass as if it has generic class decorator. We might decide to do that for all undecorated uses of MyClass or we might decide that MyClass means borrowed MyClass for formal argument types, but is generic elsewhere.
• Over on #13088, we wanted to be able to express type constraints for the generic types borrowed and unmanaged. For example in proc f(arg:borrowed), Initially arg:borrowed meant non-nil arg:borrowed and arg:borrowed? meant nilable borrowed, but then we had no way of writing a type for arg that indicates it is borrowed but with any type of nilability. We considered e.g. arg:borrowed?? for this. However, specifically for borrowed, owned, sharad, and unmanaged as generic types, we decided that we would rather have arg:borrowed accept nilable and non-nilable and then have arg:borrowed? for nilable borrowed and arg:borrowed! for non-nilable borrowed. (And similarly for owned/shared/unmanaged).

If we did choose to make MyClass / borrowed MyClass / ... have generic nilability in some situations, we would use MyClass? / borrowed MyClass? / ... to express the nilable type and MyClass! / borrowed MyClass! / ... to express the non-nilable type. We have been considering adding support for ! on types in any case as a way of getting the non-nilable type.

I can imagine 3 directions: A. MyClass means non-nilable everywhere, whether or not it allows a variety of management types (e.g. owned / shared / etc). B. MyClass means any-nilability everywhere, including in argument type declarations C. MyClass means any-nilability everywhere except for argument type declarations where it means MyClass.

A further choice we could make is to treat borrowed MyClass and undecorated MyClass differently with respect to nilability.

Additionally, for options A or C, we might want to have a way to express an argument using type MyClass but with generic nilability. This is currently possible with a where clause, but it can be important to handle it directly with a generic formal argument type in order to get appropriate coercions. If we wanted to support that, we would consider syntax ideas like MyClass??, MyClass!? or nil? MyClass.

Examples:

A: non-nilable everywhere

var x: MyClass = new MyClass(); // x has non-nilable type b/c of the type declaration
var y: borrowed MyClass = new MyClass(); // x has type non-nilable borrowed b/c of the type declaration
proc f(arg: MyClass) { ... } // arg accepts only non-nilable MyClass
proc g(arg: borrowed MyClass) { ... } // arg accepts only non-nilable borrowed MyClass

var c1: MyClass = nil;  // illegal because MyClass is non-nilable
var c2: MyClass! = new MyClass();  // legal, has type `MyClass!` == `MyClass`
var c3: MyClass? = new MyClass();  // legal, has type `MyClass?`
var c4: MyClass! = nil;  // illegal because MyClass! is non-nilable
var c5: MyClass? = nil; // legal because MyClass? is nilable

B: any-nilability everywhere

var x: MyClass = new MyClass(); // x has non-nilable type, inferred from RHS
var y: borrowed MyClass = new MyClass(); // x has type non-nilable borrowed, with nilability inferred from RHS
proc f(arg: MyClass) { ... } // arg accepts nilable or non-nilable MyClass, generically
proc g(arg: borrowed MyClass) { ... } // arg accepts nilable or non-nilable borrowed MyClass, generically

var c1: MyClass = nil;  // legal, type inferred to MyClass?
var c2: MyClass! = new MyClass();  // legal, has type `MyClass!`
var c3: MyClass? = new MyClass();  // legal, has type `MyClass?`
var c4: MyClass! = nil;  // illegal because MyClass! is non-nilable
var c5: MyClass? = nil; // legal because MyClass? is nilable

C: any-nilability everywhere except for argument declarations

var x: MyClass = new MyClass(); // x has non-nilable type, inferred from RHS
var y: borrowed MyClass = new MyClass(); // x has type non-nilable borrowed, with nilability inferred from RHS
proc f(arg: MyClass) { ... } // arg accepts only non-nilable MyClass
proc g(arg: borrowed MyClass) { ... } // arg accepts only non-nilable borrowed MyClass

var c1: MyClass = nil;  // legal, type inferred to MyClass?
var c2: MyClass! = new MyClass();  // legal, has type `MyClass!`
var c3: MyClass? = new MyClass();  // legal, has type `MyClass?`
var c4: MyClass! = nil;  // illegal because MyClass! is non-nilable
var c5: MyClass? = nil; // legal because MyClass? is nilable

[mppf]

In all of the proposals, it is expected that

var x: MyClass;

is an error. In A, it's because a non-nilable type can't be default initialized (since it can't be set to nil). In B/C, it is because a generic type (unknown nilability) cannot be default-initialized.

[bradcray]

I've been growing to like option B more and more since hearing it. I think it supports the common and preferred case cleanly (no decorator and an initialized class variable makes it non-nilable), should avoid confusion (no decorator and no initializer is a compiler error), and I like having the ability to talk about three states (definitely non-nilable, definitely nilable, could be either) using a simple and consistent syntax in all contexts.

[edited to remove proposed examples, now folded into the original issue text more cleanly]

In all of the proposals, it is expected that var x: MyClass; is an error.

In the variable declaration context, yes, but as a field, it would be legal as long as the initializer initialized x, right? (i.e., didn't rely on default initialization).

[mppf]

In all of the proposals, it is expected that var x: MyClass; is an error.

In the variable declaration context, yes, but as a field, it would be legal as long as the initializer initialized x, right? (i.e., didn't rely on default initialization).

Yes, that's right.

[mppf]

I think your examples would benefit by showing some additional cases, particularly the first two below, though I added the others for completeness:

var c1: MyClass = nil;  // illegal in A; legal in B and C, and `z` has type `MyClass?`
var c2: MyClass! = new MyClass();  // not supported in A, legal in B and C, and `z` has type `MyClass!`

I updated the issue to include these.

But, c2 is legal in A, because new MyClass() results in owned MyClass! and that is coerced to MyClass!.

[bradcray]

But, c2 is legal in A, because new MyClass() results in owned MyClass! and that is coerced to MyClass!.

Oh, I don't think I understood that MyClass! was a legal type declaration in A. Thanks for clarifying.

[bradcray]

In proposal A, it seems as though MyClass and MyClass! are synonyms for one another, is that correct? Is there a reason to support both?

[mppf]

That's right. I'd argue for supporting both because I'd view ! as a useful operator in a generic context - given a generic (nilable or not) class type, get the non-nilable class type.

[bradcray]

I see. So for cases like:

proc foo(type t) {
  var myT: t! = new t();
}

foo(MyClass?);

[lydia-duncan]

I favor option B of the proposals listed. It seems like var x: MyClass = nil; should be allowed, since the user has specified it explicitly, and we aren't forcing the user to always use either ? or !.

[bradcray]

It seems like var x: MyClass = nil; should be allowed

It is in options B and C (see the declaration of c1 in the original post), but not in option A because in that approach MyClass means the same as MyClass! (so can't store nil).

(Or are you saying this is why you prefer option B over option A?)

[lydia-duncan]

are you saying this is why you prefer option B over option A?

Yup! As for why option B over C, I think it will be simpler and don't particularly feel like we're losing anything from it.

[BryantLam]

I'm okay with the designs for A or B.

• A is similar to languages like Swift, for better or worse.
• B is similar to the proposal in unrelated issue https://github.com/chapel-lang/chapel/issues/12917#issuecomment-501442157 in that there's clear separation of generic (MyClass) and concrete syntax (MyClass! and MyClass?)
• C is potentially confusing in the same way the unrelated issues https://github.com/chapel-lang/chapel/issues/12917#issuecomment-501453092 and https://github.com/chapel-lang/chapel/issues/12917#issuecomment-501691702 note that the 2a variant (in that issue) would be an exceptional rule to the norm of generic-everywhere (in this case, B's any-nilability everywhere).

For B, is this legal?

proc B_ex1(x: MyClass) {
  x.method();
}

Does the generic deal with nil-checking the method call for me? (Maximizes user productivity at expense of compiler complexity; e.g., inserting nil-halt checks where appropriate.)

Or with the current approach to unconstrained generics, B_ex1 is generically legal for argument inputs of non-nilable ... until someone calls it with a nilable MyClass?, at which point the compiler will error out because the x.method() call is an error on nilable objects, thereby forcing users to deal with the any-nilability of x at that time.

A more practical approach might be to imitate how B_ex1 would work in a world only with constrained generics: the author of the function has to/is forced to assume that the-generic-MyClass could be nilabile and write the body as if MyClass? input was provided. If the author doesn't want to do that work of checking/unwrapping the nilable, the function signature should be explicitly written with x: MyClass! instead.

Therefore, B_ex1 should give an error message in today's implementation of something like "attempting to call method() on type MyClass that could be nilable; hint: ((do the work)) or make type MyClass!". The primary consideration for this path is future inclusion of constrained generics.

In practice, generics under B with MyClass would effectively be the same as MyClass?, but possibly with some compile-time optimizations. Thus, the syntactical inverse (MyClass == MyClass!) is proposal A (and thus why it should be more under consideration if we seriously consider the impact of constrained generics with the currently debated design.) The exception might be for container types where the difference in checking nilability at compile-time vs. run-time would actually matter.

[BryantLam]

As an aside, for proposal B, is it problematic for compile times once nilability is extended to all types and <type> is treated as generic? There's going to be a lot of generics unless folks start to adopt e.g., int! for everything. I'm also somewhat (minorly) concerned about how understandable such code will be.

[bradcray]

e.g., int! for everything.

int is not a class, so never nilable, and therefore int! and int? would never be legal, I believe.

[mppf]

@BryantLam - thanks for your thoughts.

I'm not following the connection you're seeing between this choice and the inclusion of constrained generics. Is it just that if we required constraints on all generic arguments, then it might be weird/suprising/confusing that arg:MyClass requires an additional constraint?

I have been supposing a constrained generics strategy that would not require the programmer mark all generics with constraints. Instead, the compiler would develop a set of constraints based upon what the generic function does. If this strategy were applied to your particular example, it would note that x.method() must be available and therefore at a call site B_ex1(actual), actual must be a non-nilable type.

In any case I think that this case is fundamentally different from the discussion in #12917 in that the type system has only two things to consider, for a variable with some sort of class type C: a) It can store nil b) It cannot store nil

As you said, if we have a generic nilability C, that is basically the same as C? because it can store nil. (And, as you said, the author of that generic code would need to deal with the possibility that the argument is nil in method calls etc). As far as the optimization part goes, if we wanted to, we could choose to clone functions for the assumption that arguments are not-nil and optimize them that way. We could do that with the language design A, so that seems separable.

So, put another way, one initial concern leading to B is that we are missing a clear way of writing C with any nilability. However I'm also not sure if that matters in practice - isn't it basically the same as C? since that accepts both C! and C? typed actual arguments?

This seems to me to indicate that the Swift strategy (C means non-nil, C? means nilable) might be the right choice here.

[BryantLam]

@mppf - Agreed. I think the constrained-generic argument I was making is more firm for proposal B due to the any-nilability nature of MyClass. That said, I didn't realize that part of the approach to constrained generics would be to auto-determine the constraints:

I have been supposing a constrained generics strategy that would not require the programmer mark all generics with constraints. Instead, the compiler would develop a set of constraints based upon what the generic function does.

That seems like it could be expensive / difficult to determine the constraints, but this isn't the issue for it.

In any case, I agree wholeheartedly that the issue around nilability is only really two options: is it nil or not? We don't need the generic any-nilability form if it's essentially equivalent to concrete nilable. Between A and B, I prefer proposal A since it's simpler and doesn't have any perceived disadvantages other than having to rewrite some old code, though some of that could be resolved in the interim with compiler hints/warnings.

[mppf]

I have some concern that we are herein making the opposite choice from #13088 - MyClass is non-nilable but borrowed is generic-nilability. This might be confusing.

[BryantLam]

I echo that concern. The choice in #13088 should be changed to be the same as the choice that is made in this issue, if necessary.

[BryantLam]

Throwing some ideas around. With Proposal A, how would one write a function that returns a new type that changes management type without changing the nilability.

// arg is a class with any management, any nilable.
// Aside: How to specify? `where isClass(arg)`?
proc A_RetOwned(arg) type {
  return arg.type :owned;
}

Is this correct? If so, could a reader mistake the returned type to be owned C <non-nilable> instead of the intended owned C <any-nilable>?

If I did want to change the nilability, the ! should be enough, right?

proc A_RetOwnedNonnil(arg) type {
  return arg.type :owned!; // Is this the same as `(arg.type :owned)!`?
}

In this case, the interpretation from https://github.com/chapel-lang/chapel/issues/13088#issuecomment-511415372 is that we convert to an owned, then apply ! on a possibly nilable input type to get the non-nilable output type. That all makes sense to me (in favor of Proposal A).

Anyway, I think we might need more patterns like this to figure out what pitfalls are there similar to the experiment in https://github.com/chapel-lang/chapel/issues/12917#issuecomment-498475690.

[mppf]

Just to clarify what I'm talking about as the choice from #13088 - it is this:

• borrowed means generic nilability borrowed
• borrowed! means non-nilable borrowed but is still generic over class type
• borrowed? means nilable borrowed
• similarly for owned shared and unmanaged.

Throwing some ideas around. With Proposal A, how would one write a function that returns a new type that changes management type without changing the nilability.

// arg is a class with any management, any nilable.
// Aside: How to specify? `where isClass(arg)`?
proc A_RetOwned(arg) type {
  return arg.type :owned;
}

Is this correct?

Yes, according to the above meaning of owned (generic nilability). (and answering the aside, yes isClass(arg) would do it).

If so, could a reader mistake the returned type to be owned C <non-nilable> instead of the intended owned C <any-nilable>?

Yes, if they thought that owned mean non-nilable owned, and they might think that under option A here because MyClass would mean non-nilable MyClass.

If I did want to change the nilability, the ! should be enough, right?

proc A_RetOwnedNonnil(arg) type {
  return arg.type :owned!; // Is this the same as `(arg.type :owned)!`?
}

Yes.

In this case, the interpretation from https://github.com/chapel-lang/chapel/issues/13088#issuecomment-511415372 is that we convert to an owned, then apply ! on a possibly nilable input type to get the non-nilable output type. That all makes sense to me (in favor of Proposal A).

We might consider changing the relative precedence of ! and cast.

Anyway, I think we might need more patterns like this to figure out what pitfalls are there similar to the experiment in https://github.com/chapel-lang/chapel/issues/12917#issuecomment-498475690.

Agreed! For now I am focused on the implementation undecorated-class-types-are-generic for #12917. Happy to have help coming up with interesting patterns in the meantime.

[BryantLam]

My attempt at something exhaustive using Proposal A.

//
// # Proposal A
//
// ----- Concrete -----

var a1: owned MyClass? = new owned MyClass();
var a2: owned MyClass = new owned MyCass();
var b1 = new? owned MyClass(); // Fake syntax. **Is there a way to do this?**
                               // Could propose `?` on end, but conflicts with
                               // potential use case of Optional Chaining.
                               // Hence, I went with `new?` instead, notionally.
var b2 = new owned MyClass();

proc concretefnN(farg: owned MyClass?) {}
proc concretefnX(farg: owned MyClass) {}

concretefnN(new? owned MyClass());
concretefnN(new owned MyClass());
concretefnX((new? owned MyClass())!); // Force unwrap
concretefnX(new owned MyClass());

class CompositeClass {
  var cc1: owned MyClass?;
  var cc2: owned MyClass;
}

// ----- Generic ------

var q1: MyClass? = myOwnedClassNilable;   // owned MyClass <nilable>
var q2: MyClass? = myOwnedClassNilable!;  // owned MyClass <nilable>
var q3: MyClass? = myOwnedClassNotNil;    // owned MyClass <nilable>
var q4: MyClass? = myOwnedClassNotNil!;   // owned MyClass <nilable>
var q5: MyClass? = new? owned MyClass();  // owned MyClass <nilable>
var q6: MyClass? = new owned MyClass();   // owned MyClass <nilable>

var r1: MyClass = myOwnedClassNilable;    // Illegal
var r2: MyClass = myOwnedClassNilable!;   // owned MyClass <non-nilable>
var r3: MyClass = myOwnedClassNotNil;     // owned MyClass <non-nilable>
var r4: MyClass = myOwnedClassNotNil!;    // owned MyClass <not-nilable>
var r5: MyClass = new? owned MyClass();   // Illegal
var r6: MyClass = new owned MyClass();    // owned MyClass <non-nilable>

proc factory(type t) {
    return new t();
}

var f1 = factory(owned MyClass?);
var f2 = factory(owned MyClass);
var f3 = factory(MyClass?);     // Illegal
var f4 = factory(MyClass);      // Illegal

proc semiconcretefn(arg: MyClass?) {    // arg is <nilable>
    if (arg) {
        writeln(arg);
    }
}
proc semiconcretefn(arg: MyClass) {     // arg is <non-nilable>
    writeln(arg);
}

// genericfn is tricky. I am not confident these are correct since any
// of these semantics can change via new features.

proc genericfn(arg) {   // arg MUST be a class; it can be nilable
    if (arg != nil) {
        writeln(arg);
    }
}
proc genericfn(arg) {   // arg MUST be a class; it can be nilable
    writeln(arg!);
}
proc genericfn(arg) {   // arg MAY be a class; if class, <non-nilable>
    writeln(arg);
}

// Some of these don't have mechanisms today, or I simply don't know them.

// Value cast to generic management, selected nilability
var x1 = val :(val.type?);  // <nilable> (Parens to disambig.
                            //            Precedence may change.)
var x2 = val!;              // <non-nilable>

// Value cast to selected management, generic nilability
var y1 = val :owned;        // owned MyClass <val's nilability>
    // Could be confusing, but for Proposal A, not many alternatives.

// Value cast to selected management and nilability (Parens to disambig.)
var z1 = val :(owned?);
var z2 = val :(owned!);     // Illegal? Would have to force-unwrap for values.
                            // Or I guess it could throw a NilError if nil.
var z3 = (val :owned)!;     // Today, if nil, halt.

// Type cast to generic management, selected nilability
type t1 = Ty?;              // <nilable>
    // Could be confused with value-based Optional Chaining if added later, but
    // Ty is a type so it's reasonable for types to have different rules.
type t2 = Ty!;              // <non-nilable>
    // Doesn't exist yet: type! => cast type to <non-nilable>.

// Type cast to selected management, generic nilability
type u1 = Ty :owned;        // owned MyClass <Ty's nilability>

// Type cast to selected management and nilability
type v1 = Ty :(owned?);
type v2 = Ty :(owned!);     // Cast to "owned MyClass <non-nilable>"
type v3 = (Ty :owned)!;     // Cast to "owned MyClass <generic nilable>" via u1
                            //     then cast to "owned MyClass <nilable>" via t2
                            // Same result as v2 regardless of parens.

[mppf]

@BryantLam

proc factory(type t) {
    return new t();
}

var f1 = factory(owned MyClass?);
var f2 = factory(owned MyClass);
var f3 = factory(MyClass?);     // Illegal
var f4 = factory(MyClass);      // Illegal

Following the design in #12917 (which I have working but not yet merged), I would expect the latter two of these two actually function and result on owned MyClass? and owned MyClass, respectively.

// genericfn is tricky. I am not confident these are correct since any
// of these semantics can change via new features.

proc genericfn(arg) {   // arg MUST be a class; it can be nilable
    if (arg != nil) {
        writeln(arg);
    }
}
proc genericfn(arg) {   // arg MUST be a class; it can be nilable
    writeln(arg!);
}
proc genericfn(arg) {   // arg MAY be a class; if class, <non-nilable>
    writeln(arg);
}

I would expect these to be proc genericfn(arg: borrowed?), proc genericfn(arg: borrowed?), and then proc genericfn(arg: ?t) where isNonNilable(t) || !isClass(t).

var y1 = val :owned;        // owned MyClass <val's nilability>
    // Could be confusing, but for Proposal A, not many alternatives.

We could have some other way of saying any nilability, e.g. anynil MyClass.

var z1 = val :(owned?);
var z2 = val :(owned!);     // Illegal? Would have to force-unwrap for values.
                            // Or I guess it could throw a NilError if nil.
var z3 = (val :owned)!;     // Today, if nil, halt.

For z2, I think this one throws NilError on my branch if val is nil. Note that z2 would have type owned SomeClass while z3 will have type borrowed SomeClass, because ! returns a borrow.

// Type cast to selected management and nilability
type v1 = Ty :(owned?);
type v2 = Ty :(owned!);     // Cast to "owned MyClass <non-nilable>"
type v3 = (Ty :owned)!;     // Cast to "owned MyClass <generic nilable>" via u1
                            //     then cast to "owned MyClass <nilable>" via t2
                            // Same result as v2 regardless of parens.

If we support ! on types, perhaps we would expect it to have the same result type as ! on a value with the corresponding type? In that event, ! would always return a borrow...

I'm wondering about if ! is really a reasonable choice for the non-nilable variant of a type (e.g. owned!). The reason is that the use of ! in ! and try! means "could halt". But nothing about owned! means "could halt" - in fact, because it is not nilable, there are fewer reasons for it to halt! So I think that applying ! to types is perhaps making the situation more confusing. Perhaps we need a different name for these types.

[mppf]

var b1 = new? owned MyClass(); // Fake syntax. **Is there a way to do this?**
                               // Could propose `?` on end, but conflicts with
                               // potential use case of Optional Chaining.
                               // Hence, I went with `new?` instead, notionally.

I don't see a problem conceptually with var b1 = new owned MyClass?(). But anyway even var b1 = (new owned MyClass())? I think would be fine. I am not concerned about parser conflicts between optional chaining and a ? that applies to values, because optional chaining always has a ?. which can be parsed as a totally separate token (similarly to how <= is a separate token from < followed by =).

Was there some other conflict with optional chaining you were imagining?

Edit: Here is some more example code studying the tradeoff between option A and option B:

// Assuming option 2 from #12917 (undecorated MyClass is generic management)
//
// Option A: non-nilable by default
//   * MyClass   means non-nilable
//   * MyClass?  means nilable
//   * MyClass?? means any-nilibility

// Option B: generic-nilable by default
//   * MyClass   means any-nilibility
//   * MyClass?  means nilably
//   * MyClass!  means non-nilable
//       TODO: is there some other punctuation we could use here?
//             ! means "could halt" everywhere else
//             and e.g. `owned MyClass!` isn't the same as
//             the type of ownedMyClass! because `!` returns a borrow
//             from owned/shared.

var a1 = new MyClass();        // non-nilable owned MyClass
var a2 = new MyClass?();       // nilable owned MyClass (or error?)
var a3 = (new MyClass())?;     // nilable owned MyClass

var b1: MyClass? = new MyClass();    // nilable owned MyClass
var b2: MyClass? = new MyClass?();   // nilable owned MyClass
var b3: MyClass? = (new MyClass())?; // nilable owned MyClass

// Assuming Option A: non-nilable by default
  var c1: MyClass = new MyClass();    // non-nilable owned MyClass
  var c2: MyClass = new MyClass?();   // error
  var c3: MyClass = (new MyClass())?; // error

// Assuming Option B: generic-nilable by default
  var c1: MyClass = new MyClass();    // non-nilable owned MyClass
  var c2: MyClass = new MyClass?();   // nilable owned MyClass
  var c3: MyClass = (new MyClass())?; // nilable owned MyClass

// What does ! do on values?
var d1: borrowed MyClass? = (new MyClass()).borrow();
var d2: owned MyClass? = new MyClass();

d1! // has type non-nilable borrowed MyClass
d2! // has type non-nilable borrowed MyClass

// What does ! do on types?
// It could be an error, or it could return the same type as ! on
// a corresponding value.
(borrowed MyClass?)! // non-nilable borrowed MyClass ... or error?
(owned MyClass?)!    // non-nilable borrowed MyClass ... or error?

proc factory(type t) {
    return new t();
}

var f1 = factory(owned MyClass?); // nilable owned MyClass
var f2 = factory(owned MyClass);  // non-nilable owned MyClass
var f3 = factory(MyClass?);       // nilable owned MyClass
var f4 = factory(MyClass);        // non-nilable owned MyClass

class CompositeClass {
  var cc1: owned MyClass?; // nilable owned MyClass
  var cc2: owned MyClass;  // Option A: non-nilable owned MyClass
                           // Option B: depends on how cc2 is initialized
}

// concrete function that makes a method call
proc fn1(arg: MyClass?) {
  arg!.method(); // ! necessary outside of prototype modules
}

// Accepting a non-nilable MyClass
  // Option A
    proc fn2(arg: MyClass) {
      arg.method(); // no ! necessary since arg cannot be nil
    }
  // Option B
    proc fn2(arg: MyClass!) {
      arg.method(); // no ! necessary since arg cannot be nil
    }

// Accepting any-nilable MyClass
  // The question is - is there a use case for this?
  // Why wouldn't somebody either make it totally generic,
  //  or make it accept MyClass? if they wanted to allow the possibility
  //  that it is nilable?

  // Option A
    proc fn3(arg: MyClass??) {
      if arg {
        arg!.method(); // ! necessary in general, could be instantiated nilable
      }
    }
  // Option B
    proc fn3(arg: MyClass) {
      if arg {
        arg!.method(); // ! necessary in general, could be instantiated nilable
      }
    }

[mppf]

I'd like to summarize my current position, in case it is lost in the other discussion:

from https://github.com/chapel-lang/chapel/issues/13161#issuecomment-505178089

As you said, if we have a generic nilability C, that is basically the same as C? because it can store nil.

Another way to put it is that MyClass being generic nilibility is that it makes the common case actually a case that I don't know if there is any use case for.

I updated https://github.com/chapel-lang/chapel/issues/13161#issuecomment-514766693 to show some examples... but if one is writing a function e.g. proc f(arg: MyClass) if you wanted it to handle the possibility that it's called with nil or not, wouldn't you write just the nilable version with proc f(arg: MyClass?) anyway?

from https://github.com/chapel-lang/chapel/issues/13161#issuecomment-514765066

I'm wondering about if ! is really a reasonable choice for the non-nilable variant of a type (e.g. owned!). The reason is that the use of ! in nilableThing! and try! means "could halt". But nothing about owned! means "could halt" - in fact, because it is not nilable, there are fewer reasons for it to halt! So I think that applying ! to types is perhaps making the situation more confusing. Perhaps we need a different name for these types.

Also I want to clarify something from the above exchange between @BryantLam and @bradcray :

e.g., int! for everything.

int is not a class, so never nilable, and therefore int! and int? would never be legal, I believe.

Swift views MyClass? as Optional(MyClass) where Optional is a generic union/enum type containing nil or a value. It seems to me that this strategy allows for some useful language features. In particular, it seems hard to imagine how Optional Chaining could work without it. In particular, without the viewpoint that MyClass? is the same as Optional(MyClass), I can't imagine that optional chaining could work with a method that returns a non-class type such as int. How would it represent the lack of an int in that case?

Note that in the current nilable types proposal #12614:

• we expect to eventually support optional records / optional int
• we expect to eventually support optional chaining - something along the lines of this:
  • possiblyNil?.someMethod() returns nil if possiblyNil stores nil, and otherwise wraps the result of someMethod in an optional.

The point is that the Optional(Something) interpretation is an important generalization that allows reasonable further language features. But in that viewpoint, it seems unnatural and conceptually confusing for the default meaning of SomeType to be "maybe Optional(SomeType) and maybe just SomeType". Because what is "just SomeType"?

For these reasons, I continue to be unhappy with MyClass! meaning the non-nil variant and with MyClass being generic nilability.

[lydia-duncan]

Michael talked to me about this, and I'm feeling convinced by his arguments. What helped me get swayed was realizing that I was still conflating nil as a value for classes (and so was really viewing undecorated MyClass as equivalent to MyClass?), when part of this concept is very much separating nil as a value from the other potential values of MyClass. That hadn't really clicked for me before.

[vasslitvinov]

Michael hasn't talked to me about this, so I'm feeling unconvinced (yet). :)

I am still charmed by the beauty of uniformity where MyClass means generic in all dimensions. So these forms adjust the variables' types to their initializers:

var c0 = rhs;
var c1: MyClass = rhs;            // uses rhs's nilability and memory mgmt
var c2: owned MyClass = rhs;      // uses rhs's nilability
var c3: MyClass! = rhs;           // uses rhs's memory mgmt
var c4: owned MyClass? = rhs;     // owned, nilable
var c5: owned MyClass(int)? = rhs;  // also, rhs must be a correct instantiation

The same goes for formal arguments.

As to the question "if the user wants to write a function that can handle both nilable and non-nilable types, why not use arg:MyClass? right away?": Obviously, a nilable arg and a generic-nilabiity arg differ in how many runtime checks are executed when the actual argument is non-nilable. Less obviously, in my experience converting existing code in #13397, the cases where the formal needs to be nilable are more frequent than I had expected. So the mix of nilable and non-nilable formals is more even. Choosing between these two options:

// Option A
proc myFun1(arg: MyClass) ...
proc myFun2(arg: MyClass?) ...

// Option B
proc myFun1(arg: MyClass!) ...
proc myFun2(arg: MyClass?) ...

Option B provides a lower learning barrier for people to grasp the difference when they see a mix of the two syntaxes in the code.

Perhaps this brings me back to my original motivation: it is easier to remember that MyClass means generic everything than "generic memory management, non-nilable".

By the same consistency argument, owned, shared etc. by themselves should have generic nilability.

Another thought: with Option B, when I am writing a function

proc myFun(arg: MyType) ...

in the "rapid prototyping" mode, I can assume MyType is non-nilable. This will work just fine, as long as I do not invoke myFun on a nilable. Or as long as the body does not require non-nilability. This allows me to postpone the decision "nilable or non-nilable?" until it actually matters.

Living in the world where int? means Optional(int) and int means "generic over Optional or not", I can keep using arg: int in the generic sense as long as no issues come up. Although this is more of a stretch because "generic over Optional or not" for integers is unlikely to be the common case. Assuming that Optional(int) has performance overhead over pure ints.

[vasslitvinov]

In my perception !-could-halt and !-type-modifier are from different domains. Therefore I am fine with ! having distinct meanings in the two contexts.

This expression from an earlier example:

    ..... myClass:borrowed! .....

is surely tricky. On the one hand, to me ! binds to the type more tightly than the cast operation. So intuitively I interpret it as:

    ..... myClass:(borrowed!) .....

However, this operation is not very helpful, as it produces a non-nil when myClass is non-nil and nil when myClass is nil. So we should perhaps disallow this at compile-time when myClass.type is nilable. Instead the user would have to parenthesize explicitly:

    ..... (myClass:borrowed)! .....

I also propose that in the following:

var d2: owned MyClass? = new MyClass();
..... d2! .....

d2! has the type owned MyClass! and behaves as an alias to d2 itself, with a runtime check added and the static type treated as non-nilable. That way d2! will release ownership - or not - just like d2 would.

In other words, it seems natural to me that ? and ! postfix operators retain the lvalue-ness of the argument.

I can see how Swift would make a different choice here. For example, it is not sound to alias an int and an Optional(int) if Optional is implemented as a record or a union. However, for us when d2 is a class, aliasing is sound when the runtime check succeeds.

Providing aliasing for !/? on classes, however, takes us farther away from introducing Optional on non-class types in a cohesive way. So does treating of MyClass as generic over nilability, which would probably not extend naturally to ints (see my previous comment).

[vasslitvinov]

Michael hasn't talked to me about this, so I'm feeling unconvinced (yet). :)

Now having talked to Michael :) I am OK with the default-nonnilable route.

While my considerations above have not changed, the following factors in support for default-nonnilable stand out for me:

• Generic nilability is unlikely to be the common case in practice, so it is not wise to make it the syntactic default.

• Since some of us think that we will have Optional for non-class types, like integers, we want to plan for that now. In particular, avoid the slope towards int syntactically meaning "generic over Optional-ity".

[bradcray]

Having caught up, I'm on board with Michael's current proposal.

[mppf]

I'm closing this issue since I believe there is consensus for Option A for the reasons described above. This is what is currently implemented.