Consider disallowing none to be stored in Array, Dict or any other future collections type

[masak]

Java had this interesting thing that they learned with the collections API, a bit too late — as far as I understand, it really hit home when they were doing the concurrent collections like ConcurrentHashMap: null is not a good citizen when it comes to being an element, a key, or a value.

(Why? I guess the reasoning with concurrency goes something like this: every time we promise to handle null, that leads to handing a special case. Speacial cases means if statements, and that means either check-then-act (not so good for concurrency) or more locking (not so good for performance). That's just a guess, though.)

But maybe the lesson holds in general, and maybe it's a good idea to just implement preemptively for all collections: none and collections don't mix, on general principle.

I can see some reasons to want to use none as a kind of sentinel, but those can all be rephrased as using some other value as a sentinel instead. false, for example, or something more type-specific.

Some coding idioms become nicer and cleaner from this. For example, the method corresponding to Python's get could return none and we'd be sure that means "key not found".

[vendethiel]

For dicts, that'd only entail a restriction on the keys, right?

[masak]

I thought so too, but no. See ConcurrentHashMap for example; it disallows null as both keys and values.

I think this is because the "Some coding idioms become nicer and cleaner from this" reason I gave. Basically, what they seemed to have realized that this is a breach of the Semipredicate problem that they need to kill off at put-time so it doesn't become an issue at get-time.

[masak]

In the Map interface in Java 8, the new methods compute, computeIfAbsent, computeIfPresent, and putIfAbsent all treat a passed or computed null value as meaning "make there not be an entry here". Either by removing an existing one or just not putting in a new one.

I think I like the API, but I'm also a little... surprised... to see it in a Java API. It feels like a genuinely Perlish idea, to (a) have a set of methods do insertion on paper, but then also throw in deletion as a bonus, and (b) to do the latter using a special Bottom-like out-of-band value which no-one likes and which its inventor calls a very expensive mistake.

Even funnier, these default implementations on Map are then also available on e.g. HashMap, which predates the realization that nulls don't belong in compound data structures. So you can put a null in a HashMap; you just can't do it with these new methods.

[masak]

I was thinking about the case of a metacircular runtime (#51) and how this issue would interact with it.

Implementing Array and Dict using Array and Dict (respectively) is no big problem. But lexical pads (which hold variables and their values) would presumably be implemented using Arrays or Dicts — and these do have to represent none values, for uninitialized variables, etc.

My first thought was to have a special sentinel value on the host level, and then intercept on the way in and out of data structures. But this both has a performance overhead, and feels like it would be hard to make airtight. (Setting yourself up for eternal vigilance is not a super idea.)

So how about this: the guest 007 gets its own none value. Since it's not the none value of the host, it can be stored in arrays and dicts. There's kind of a neat layering going on: guest-none is slightly more permissive than host-none. We'd still have to put in explicit checks against storing guest-nones in user-facing arrays and dicts, but we no longer have to intercept anything on the way out. I think this is a good idea all the way.

[masak]

I was thinking how, since none is not a bottom value like null is, this issue might actually be a bad fit for 007's type system.

An array gets typed to contain only Str elements. Well, it already isn't allowed to contain a none, since there is no type compatibility.

How would we write out the type for an "untyped" array? It's allowed to contain anything (so Object, or any)... except for none? Type subtraction?

[masak]

Going to close this one. It's an interesting idea, and one that a language with goals other than 007's might pursue. For 007, this feature would constitute "grinding an axe" (i.e. foisting an ideological opinion on users), and without any support from either Perl 6 or Python.