PowerShell / PowerShell

PowerShell for every system!
https://microsoft.com/PowerShell
MIT License
43.66k stars 7.08k forks source link

Suggestion: implement null-coalescing, null-conditional access (null-soaking), null-conditional assignment #3240

Closed mklement0 closed 4 years ago

mklement0 commented 7 years ago

Null-coalescing and null-conditonal access (null-soaking) would be handy additions to the language.

Update: @BrucePay additionally suggests null-conditional assignment, with ?= - see comment below.

For instance, instead of writing:

if ($null -ne $varThatMayBeNull) { $varThatMayBeNull } else { $fallbackValue }
#
if ($null -ne $varThatMayBeNull) { $varThatMayBeNull.Name } else { $null }

one might be able to write:

$varThatMayBeNull ?? $fallbackValue  # null-coalescing
# 
$varThatMayBeNull?.Name   # null-conditional access, for Set-StrictMode -Version 2+
$varThatMayBeNull?[42]  # ditto, with indexing

Re null-conditional access: With Set-StrictMode being OFF (the default), you can just use $varThatMayBeNull.Name - no special syntax needed; however, if Set-StrictMode -Version 2 or higher is in effect, $varThatMayBeNull.Name would break and that's where the null-conditional operator (?.) is helpful, to signal the explicit intent to ignore the $null in a concise manner.


Open question:

$varThatMayBeNull?[42] handles the case where the variable is $null, but if it isn't, an array element with the specified index must exist.

It would therefore also be helpful to make indexing null-conditional - something that C# does not support, incidentally (you have to use .ElementAtOrDefault()).

The two basic choices are:


Related: implement ternary conditionals

iSazonov commented 6 years ago

@mklement0 It seems the second sample works by design without "?". Maybe change to name()?

mklement0 commented 6 years ago

@iSazonov: Good point, but it only works without the ? unless Set-StrictMode -Version 2 or higher is not in effect - I've updated the initial post to make that clear.

Halkcyon commented 5 years ago

I think syntactic sugar is something powershell could use that other languages enjoy. +1 on the proposal.

kfsone commented 5 years ago

Consider the following samples from other languages:

a="${b:-$c}"
a = b || c;
a = b or c
a := b ? c
a = b ? b : c;

even

a = b if b else c

is better than

if (b) { a = b } else { a = c }

and often times you need to clarify with the additionally verbose

a=(if (b -ne $null) { b } else { c })

Makes one feel all dirty and bashed.

thezim commented 5 years ago

Found myself doing this today as work around.

$word = ($null, "two", "three").Where({$_ -ne $null}, "First")
bgshacklett commented 5 years ago

I use a similar pattern:

$word = ($null, "two", "three" -ne $null)[0]
BrucePay commented 5 years ago

@kfsone There is a small error in your example $a=(if ($b -ne $null) { $b } else { $c }). It should be $a=$(if ($b -ne $null) { $b } else { $c }) however the only version of PowerShell where $( ) was required was version 1. From v2 on, you can simply do:

$a = if ($b) { $b } else { $c }
Halkcyon commented 5 years ago

@bgshacklett Don't you mean $word=($null,'two')[$null -ne 'three']?

It's a bit unfortunate this has gone from 6.1 to 6.2 to "Future".

bgshacklett commented 5 years ago

@TheIncorrigible1 No, if you copy and paste what I added above, you should see that the value of $word is set to "two". There's more detail in this Stack Overflow answer where I first saw the pattern: https://stackoverflow.com/a/17647824/180813

kfsone commented 5 years ago

While perlesque hack-arounds appeal to the 20-year ago me that wrote a shebang that registered and/or queried a RIPE-DB user or organization record, what I'm hoping for from powershell is something that encourages my colleagues to use the language rather than instill fear in them.

My litmus test is this: Would I want to read this via my tablet at 3am New Years morning while hung over with the ceo on the phone crying at me how many millions of dollars we are losing a second.

(Aside: this was only a loose exaggeration of an actual experience until I worked at Facebook and came back from an urgent quick leak to be told that, in the 2 minutes I was gone, more people than the population of Holland had gotten an empty news feed. It wasn't my code and the mistake was a minute change in the semantics of return x vs return (x) in the c++ standard for very specific template cases, and "would you want to read this code with a 2 minute deadline, a full bladder and the fate of every clog wearing Dutch person's cat pictures on your shoulder???" didn't sound as cool)

vexx32 commented 5 years ago

Yup. There are some neat tricks possible in PS that are great in a pinch or if you need a one-time shorthand.

For maintainable code, ideally we should have explicit null-coalescing operators like C#. The main trouble there for me is -- what do we use for those? ? is already aliased to Where-Object (much as I'd love for that to be erased, it's in very common use). Mind you, % is aliased to ForEach-Object but that doesn't hinder modulus operations, so in theory at least having ? be a null-coalescing operator as well would potentially be fine; it would only be interpreted as such in expressions, where Where-Object isn't really valid anyway.

mklement0 commented 5 years ago

@vexx32:

At least syntactically ?? should be fine, because we're talking about expression mode, whereas ?, as a command [alias], is only recognized in argument mode. Not sure if reuse of the symbol causes confusion, but it wouldn't be the first time that a symbol does double duty in different contexts.

Surprisingly, using ?. and ?[] for null-soaking would technically be a breaking change, because PowerShell currently allows ? as a non-initial character in a variable name.

PS> $foo? = @{ bar = 1 }; $foo?.bar   # !! $foo? is a legal variable name
1

However, I hope this would be considered a Bucket 3: Unlikely Grey Area change.

vexx32 commented 5 years ago

I can't say I've ever seen anyone use ? in a variable name... Nor would I, because chances are it would be misread. But yeah, hopefully that should be perfectly useable.

bgshacklett commented 5 years ago

I suspect it's a bit late to bring this up, but Bash has handled this (and some other cases) with its Parameter Substitution features: https://www.tldp.org/LDP/abs/html/parameter-substitution.html.

While it can be a bear to learn due to the sheer number of things that can be done with it, it's incredibly powerful. I understand that it would not be possible to use this exact notation due to the way PowerShell uses braces with variables, nor would it necessarily fit with the general feel of the language, but it seems like a useful data point.

mklement0 commented 5 years ago

@bgshacklett:

Yes, parameter substitution is powerful, but, unfortunately, it's not only a bear to learn, but also to remember.

So while Bash's features are often interesting, their syntactic form is often arcane, hard to remember, and not a good fit for PowerShell.

Brace expansion (e.g., a{1,2,3} in bash expanding to a1 a2 a3) is an example of an interesting feature whose expressiveness I'd love to see in PowerShell, but with PowerShell-appropriate syntax - see #4286

bgshacklett commented 5 years ago

I completely agree. I brought it up more as an example of how this issue has been solved elsewhere than an exact solution.

BrucePay commented 5 years ago

There's one other operator to possibly consider:

$x ?= 12

which would set $x if it's not set (doesn't exist). This is part of the "initializer pattern" which is not common in conventional languages but for (dynamically scoped) languages like shells, make tools, etc. it's pretty common to have a script that sets a default if the user hasn't specified it. (Though in fact parameter initializers are pretty widespread.)

Extending this to properties:

$obj.SomeProperty ?= 13

would add and initialize a note property SomeProperty on the object if it didn't exist.

And - for fun - one more variation on initializing a variable using -or:

$x -or ($x = 3.14) > $null
mklement0 commented 5 years ago

$x ?= 12 sounds like a great idea.

It occurred to me that we probably should apply all of these operators not only if the LHS doesn't exist, but also if it does exist but happens to contain $null (with [System.Management.Automation.Internal.AutomationNull]::Value treated like $null in this context).

add and initialize a note property SomeProperty

In that vein, $obj.SomeProperty ?= 13 makes sense to me only if .SomeProperty exists and contains $null, given that you cannot implicitly create properties even with regular assignments (by contrast, for hasthables the implicit entry creation makes sense).

All operators discussed will need to exempt the LHS from strict-mode existence checking.

HumanEquivalentUnit commented 5 years ago

I'm wondering about the intent / reasoning behind why strictmode stops you from accessing a property which doesn't exist. Is the intent to catch typos like $x.pretnd? Is the intent to check assumptions like assert(x != null)? Is the intent to block accidentally propagating $null further on in your code?

the null-soaking (?.) is helpful, to signal the explicit intent to ignore the $null in a concise manner.

Is that not what . signals already in nonstrict mode? If you had wanted to be strict, you could have checked for .Name existing first, by not checking first, you already signalled explicit intent that checking wasn't important to you.

If StrictMode has a purpose where accessing a non-existent property and returning $null is forbidden, then won't the same reasoning apply to ?., and that should throw the same exception for the same reason. If not, why not?

With null-coalescing, you could write:

$result = Invoke-RestMethod -Uri 'https://example.org/api/test'
$test = $result?.Name

Will it quickly become recommended or idiomatic to always use ?. for every property reference, because it "behaves like . does, and won't throw exceptions for people using strictmode"?

Is the real desire to have a configurable strictmode with variable settings, so that e.g. you could have a declaration at the top of your script # StrictMode SkipMemberExistenceCheck, or an escape hatch attribute for indicating a block of unstrict code generally? [I suspect the desire is terse C# style syntax, but you know what I mean]

mklement0 commented 5 years ago

Thanks for the thoughts, @HumanEquivalentUnit.

Is the intent to catch typos like $x.pretnd?

I can't speak to the design intent, but that's what makes sense to me - analogous to what Set-StrictMode -Version 1 does for variables (only).

With variables it's about their existence, not whether the value happens to be $null, and checking for property existence follows the same pattern.

I think of it as the closest a dynamic scripting language can get to finding errors that a statically typed compiled language would report at compile time.


Is that not what . signals already in nonstrict mode? If you had wanted to be strict, you could have checked for .Name existing first

Yes, but in non-strict mode you forgo the aforementioned existence checks, and testing manually is obviously quite cumbersome - and invariably slower than a built-in feature.

by not checking first, you already signalled explicit intent that checking wasn't important to you.

The point is that, with Set-StrictMode -Version 2 or higher, you get:

you could have a declaration at the top of your script # StrictMode SkipMemberExistenceCheck, or an escape hatch attribute for indicating a block of unstrict code generally

Related: The current dynamic scoping of Set-StrictMode is problematic; see @lzybkr's RFC draft for implementing a lexical strict mode: https://github.com/PowerShell/PowerShell-RFC/blob/master/1-Draft/RFC0003-Lexical-Strict-Mode.md

HumanEquivalentUnit commented 5 years ago

the benefit of member existence checking by default

PS contains code which does member checking, and returns $null when they don't exist. That's a benefit by default, which you can also avoid by writing your own member existence test with $x.psobject.properties... if you choose to. StrictMode takes away that benefit and leaves the hinderence of having to write existence tests. Then instead of giving a nice way to write exactly a member existence test, ?. would give you a concise workaround to get what . always did. And if it matters that the member exists, you still have to code that separately.

But this does make me ask what about method calls:

$x.pretend     -> $null
$x.pretend()   -> Exception MethodNotFound
$x?.pretend  -> $null
$x?.pretend()   -> ____ what goes here? $null?

set-strictmode -version 2
$x.pretend -> Exception PropertyNotFoundStrict
$x.pretend()  -> Exception MethodNotFound
$x?.pretend  -> $null
$x?.pretend()   -> ____  ?

i.e ?. would behave the same as . for property lookups in non-strictmode, but would behave differently to . for method calls in non-strictmode, making it nuanced and not a drop-in replacement. Does that mean that when using ?. you would have no way at all to tell whether a method was invoked, or nothing at all happened? Properties can be backed by getter methods, but I think it's convention in DotNet for getters not to mutate internal state, but normal for method calls to mutate state. With ?. you would have no idea if you mutated an object's state?

I wonder if you can get the desired benefits, and better, by making ?? able to handle PropertyNotFoundException and MethodNotFound Exceptions directly from the expression on the left (but not other Exceptions raised inside methods), as well as handling $nulls, and not have ?. at all. e.g.

$varThatMayBeNull ?? $fallbackValue # null-coalescing
#
$varThatMayBeNull.Name ?? $fallbackvalue    # catching PropertyNotFoundStrict exception
$varThatMayBeNull.Name ??  # default fallback is $null
$varThatMayBeNull.Method() ??  # catching MethodNotFound Exception

# and potentially addressing the index case too
$varThatMayBeNull.first.second[3].Method() ??  # catching MethodNotFound Exception, or index not found Exception

And then, instead of ?. calling a member or accessing a property, it could be a concise way of asking if a member exists, returning a [bool].

With the same syntax addition of ?? and ?. you could handle more situations, with less confusing overlap.

mklement0 commented 5 years ago

StrictMode takes away that benefit and leaves the hindrance of having to write existence tests

One person's hindrance is another person's benefit: It all depends on what you want to have happen by default:

Explicitly testing for the presence of a member is really a separate use case that can be a necessity irrespective of the strict mode in effect.

mklement0 commented 5 years ago

As for method calls, which the OP doesn't cover:

Method calls (e.g., $varThatMayBeNull.Method()) and indexing (e.g., $varThatMayBeNull[$index]) are two cases where even strict mode being off or at version 1 could benefit, given that both cases currently invariably cause an error if the value being accessed is $null.

The syntax would be same as for properties for methods - $varThatMayBeNull?.Method() - and analogous for indexing - $varThatMayBeNull?[$index] - as in C#.

Update: There is a second aspect to indexing, namely if the variable is non-null, but the element at the given index doesn't exist (e.g., $arr = 0,1; $arr[2]). With strict mode version 2 or below, this evaluates to $null, but in version 3 or higher it causes an error, and it would be nice to be able to opt out of this error as well. However, the syntax for this case presents a challenge - see the updated OP, which currently proposes the perhaps awkward [...?].

And, yes, I would make such access default to $null as well, even for methods - again, you're explicitly signaling that it's fine to do nothing if the object being accessed is $null or if no such array element exists.

Note that C# has no null-conditional indexing in the second sense.

vexx32 commented 5 years ago

It has null-conditional indexing, just not quite like how you're proposing. C# allows (I'm pretty sure, at least...) array?[0] which bypasses the usual "index into null array/collection" issue and just returns null.

But yeah, I agree that this is a good idea. It saves us going through exception handling, and it keeps code pretty clear and concise.

mklement0 commented 5 years ago

@vexx32, that is one aspect of null-conditional indexing: if array in array?[0] is null, the indexing operation is ignored - C# has that too, as you state.

I was (update: also) talking about the other aspect of null-conditional indexing: the case where array in array?[0] is not null but it is the element at index 0 that doesn't exist.

It is the latter that cannot be ignored in C#, to my knowledge, yet I think it would be useful too.

Can you think of terminology to give these two aspects distinct names?

HumanEquivalentUnit commented 5 years ago

The code from another JavaScript issue would go from $_.Description[0].Text to $_?.Description[0?]?.Text which I think is not pretty; and my other suggestion for how ?? could behave would take it to $_.Description[0].Text ?? which is subjectively nicer, and would - if possible to implement - handle failures at any lookup point in the chain all in one go, and still let you choose what you want to happen by default, and have a different fallback at will instead of being limited to only $null.

As an aside, would this ?. syntax have an analogue for static properties and methods, i.e. $t = [int]; $t::MaxValu; $t::Pars("1"), would that become ?: or ?:: ?

What would be the desired behaviour for array indexing with more than one index specified? $array[0,4?,1,2] where each one can be allowed to fail, or $array[0,4,1,2?] with only the entire lot allowed to succeed or fail?

mklement0 commented 5 years ago

I first have to take a step back, because I realize that I've conflated aspects that are worth separating, at least conceptually:

Note: For brevity I'm using member somewhat loosely here to also encompass elements of objects that happen to be collections, which are accessed with indexing rather than dot notation.

HumanEquivalentUnit commented 5 years ago

If you're confident that there are no typos in your code - by all means, take advantage of the default behavior (strict mode OFF) - and you'll have no need for .? (for property access

@mklement0 I think you're discussing from the view "one person using strict mode to improve their own code" and I from the view "someone else can run my code in strictmode, what changes would make my code work for them as well?" and that's leading me down different reasoning. With ?. it would be so convenient to use it instead of . and make code also work in strictmode that I may as well use it everywhere habitually; I lose nothing and gain compatibility. I fear all PowerShell code will uglify in that way because it "works" in more cases.

But it isn't a drop-in replacement, because it silences failed method call lookup in non-strictmode, and silences failed property lookups in strictmode - it adds a fiddly nuance to member lookup, an incredibly common operation. It doesn't help anyone who wants to benefit from strictmode checks by making that boilerplate easier. It doesn't help enough people in enough situations, for the complexity it adds. And it's too convenient to make code more compatible, I don't think "ignore it if you don't want it" is strong enough to hold it at bay.

PowerShell can do things C# doesn't, like setting automatic variables, or having a precedent for having exceptions be thrown and silenced behind the scenes. There's got to be a better approach which doesn't complexify member lookup at all, but simplifies strictmode safety check boilerplate so that people who do want to use the checks have easier lives and people who were annoyed by the checks have easier lives as well.

Halkcyon commented 5 years ago

@HumanEquivalentUnit How does it silence these things? You would have to write a load of exception handling and null checks previously, instead:

$prop = $item?.Method().PropIwant ?? 'PropActionFailed'

v.

$prop = if ($null -ne $item) {
    $propIwant= $item.Method().PropIwant
    if ($null -eq $propIwant) {
        'PropActionFailed'
    }
    else {
        $propIwant
    }
}

And this case isn't uncommon. Or you just throw $ErrorActionPreference = 'Stop' and wrap the whole thing in try/catch to simplify the null checking which is a bad pattern (exception-driven logic).

Also, this is the future of the language, it's not like we're going back to 5.1 and changing things. If you want more adopters, you need to make it easier to read code, which I feel this suggestion does.

kfsone commented 5 years ago

@mklement0 I think you're discussing from the view "one person using strict mode to improve their own code" and I from the view "someone else can run my code in strictmode, what changes would make my code work for them as well?"

Are we talking about "set -euo pipefail" (bash strict mode) equivalent?

a- Limit to variable look-ups (although I don't see why you think a simple null check on member lookups would be a bad thing, the actual member lookup is way more expensive than if (result is null)) b- Use [Strict] on scopes/codeblocks to tell the parser to treat them as such, c- Strictness is based on scope: code that isn't marked strict isn't strict.

If I wrote strict code that fails when you use it, then one of us made a mistake. Either I referenced a missing variable in code I never tested or you've misunderstood my badly designed API that depends on the existence of variables that aren't passed in to my function and you failed to populate a variable I expect to be populated...

# Their code
Function Log($Message) { Write-Host $Massege }  # not strict

# My code
[Strict]
Function DebugMsg
{
  Param([String] $Message)
  if ($DEBUG) { Log Host $Message }
}

# Your code
DebugMsg "Hello"  # you didn't define DEBUG, and by saying strict I expressed a contract whereby the variable MUST be defined.

The "Log" function wasn't marked strict so it doesn't complain about the typo

vexx32 commented 5 years ago

πŸ€” Now that you mention it, a [Strict()] attribute would be an awesome way to implement strict mode in a scope-restricted way. That needs to be mentioned in that other issue thread on strict mode issues, I'll go dig that up and link back here...

HumanEquivalentUnit commented 5 years ago

@TheIncorrigible1 "How does it silence these things?" in non-strict mode $null.x is silent, $null.x() is an exception. With ?. as discussed, $null?.x is silent, $null?.x() is silent.

So you'd have a . which works like C#'s ?., except in strict-mode where it throws exceptions. Then you'd have a ?. which works in strict mode like C#'s ?. and like PowerShell's . works in in non-strict mode, but which works differently to PowerShell's . in non-strict mode for method calls. What a horrible combination, (and a combination where ?. is clearly better and more useful in more of those scenarios).

You would have to write a load of exception handling and null checks previously, instead: $prop = $item?.Method().PropIwant ?? 'PropActionFailed'

Your method call might throw, or return $null and PropIWant might be missing, you'd still have to write:

$prop = try {
    $item?.Method()?.PropIwant ?? 'PropActionFailed'  
} catch [someMethodException] {
    'PropActionFailed'
}

vs with my proposed ?? (if it was possible to build), and no need for ?., and it covers possible cases of Method exceptions in case all you care about is "success or not" and no detailed error handling.

$prop = $item.Method().PropIWant ?? 'AnythingFailed'

If you want more adopters, you need to make it easier to read code, which I feel this suggestion does.

How often do you type . in PowerShell? How often do you want to explain to new users what the difference is between . and ?. and tell them they have to consider the difference every time they type a . forever?

"Oh, how did that come about?" "Well in C# . threw an exception for missing members, so they invented ?. which didn't. That was great, so they did that in PowerShell by default, the . doesn't throw. Then they invented strict-mode which makes . behave like C# and throw but only in that context, But the way to manually check members was really wordy and bothersome and nobody wanted to type it; instead of addressing that problem head-on and making that more convenient to benefit from the strictmode, they dodged that and aped C#'s ?. instead to gain a sneaky way out of the strictness that you asked for, which is mildly convenient for you, but sucks for everyone else. And you still have no member existence test, and benefiting from strictmode is still wordy and bothersome, but at least you don't have to suffer that because 'strictmode' has a 'normalmode' escape hatch now". Ick.

If I wrote strict code that fails when you use it, then one of us made a mistake.

although I don't see why you think a simple null check on member lookups would be a bad thing, the actual member lookup is way more expensive than if (result is null)

I don't think that would be bad, PS non-strictmode already does that, and I think that would be a desirable alternative to the proposed ?. - very different, useful in more situations.

But note that in PS you can do something C# cannot:

(1..3)[1kb..100kb]

and no exceptions in non-strictmode. Change the numbers and see how long it takes; judging by the performance and how I think it must work behind the scenes to index into arbitrary objects, it seems there actually are ~100,000 exceptions being raised and silenced, for our convenience. PS doesn't always make the "must be as fast as possible, push the inconvenience on the user" choice, and I like that about it.

Halkcyon commented 5 years ago

@HumanEquivalentUnit There are many features in every language you may never use in your career and if you don't like the style, that's fine, you don't need to use those features. I don't see why you're writing strict mode off; it's a good practice in scripts so you're consciously dealing with problems rather than letting the language swallow them (and in essence, having implicit empty catch blocks everywhere). Again, you're still welcome to not use the features (e.g. I've never used Out-Default).

Also, the operator is ?, not ?.; it would work with index access as well.

mklement0 commented 5 years ago

To continue to lay the groundwork for further discussion, let me summarize how the existing strict modes affect null-value-conditional and member-existence-conditional access (the terms introduced above):

The columns below represent the Set-StrictMode settings, and the column values indicate:

As an aside: the fact the errors are only statement-, not script-terminating, relates to the larger discussion around PowerShell's error handling - see https://github.com/PowerShell/PowerShell-Docs/issues/1583.

Construct -Off -Version 1 -Version 2 -Version 3+
$obj πŸ‘ 🚫 🚫 🚫
$obj.Prop πŸ‘ πŸ‘ 🚫 🚫
$obj[42] 🚫 🚫 🚫 🚫
$obj.Method() 🚫 🚫 🚫 🚫

It is curious that with -Off and -Version 1, $obj.Prop is allowed, whereas $obj[42] isn't.

Construct -Off -Version 1 -Version 2 -Version 3+
$obj.Prop πŸ‘ πŸ‘ 🚫 🚫
$obj[42] (indexable) πŸ‘ πŸ‘ πŸ‘ 🚫
$obj[42] (non-indexable) πŸ‘ πŸ‘ 🚫 🚫
$obj.Method() 🚫 🚫 🚫 🚫

It is curious that inherently indexable objects (e.g., arrays) allow access to nonexistent elements even with -Version 2, whereas scalars - where PowerShell provides indexing capabilities for the sake of unified handling with collections - do not.

IanKemp commented 4 years ago

The null-soaking (null-conditional or Elvis in C#) operator is almost mandatory when running in StrictMode Latest and trying to access a property that may or may not exist. Currently you have to resort to stupid tricks like this:

function Get-OptionalPropertyValue($object, [string] $propertyName) {
    if (-not ([bool] (Get-Member -InputObject $object -MemberType Properties -Name $propertyName))) {
        return $null
    }

    $object.$propertyName
}

$value = Get-OptionalPropertyValue $foo "bar" # if $foo.bar exists, $value will contain its data; else $value will be $null

when you really should just be able to write:

$value = $foo?.bar # if $foo.bar exists, $value will contain its data; else $value will be $null

Please guys, if you could get only null-conditional from this proposal implemented and only for properties, it would save so many people so much pain.

adityapatwardhan commented 4 years ago

I started having a look at this and found a few problems.

Lets consider ?? and ?=

$x ?= 1 and $x ?? 1 seem to be straightforward. But, since they are operators we need to support $x?=1 and $x??1.

The problem with that is $x? and $x?? both validate variable names in PowerShell. The only way to disambiguate would be ${x}?=1 and ${x}??1.

And more over for conditional member access ?. and ?[], we would have to have ${x}?.name and ${x}?[0]. For these, $x ?.name and $x ?[0] will not be supported.

The other option is to introduce a breaking change and not allow ? in the variable name at all.

What do you think? @mklement0

/cc @rjmholt @daxian-dbw @JamesWTruher

vexx32 commented 4 years ago

I have never particularly liked nor understood why ? is a valid variable name. It opens the door for names like $Safe? which is generally more clear and less awkward when written as $IsSafe anyway.

I think that change has probably been a long time coming.

rjmholt commented 4 years ago

It opens the door for names like $Safe? which is generally more clear and less awkward when written as $IsSafe anyway

Interestingly I expressed the opposite sentiment. It's a classic Scheme/Ruby tradition.

if ($questionMark?)
{
    Write-Host "That's silly!"
}
KirkMunro commented 4 years ago

Didn't the ternary operator have this same problem, where the resolution was to require spaces around the operator or curly braces around the variable name in order for it to properly work with variables that end with ??

I would expect that with these operators being introduced, PowerShell would prioritize recognizing the ? as part of the operator in syntax such as $x?[0], such that users who use variable names ending in ? to store a collection would have to use this syntax instead (since spaces would be invalid in this case): ${x?}?[0]. Ditto for $x?.Name and ${x?}?.Name.

rkeithhill commented 4 years ago

The other option is to introduce a breaking change and not allow ? in the variable name at all.

πŸ‘ Given this is a major version change (6 to 7), I think it is worth a breaking change here. I wish GitHub had better code search to see how many instances there are of PS variables that end with a ?. I suspect this wouldn't be that impactful but it would be nice to verify that against GitHub. In 14 years of using PowerShell, I've never used a ? in a variable name but maybe that's just me. :-)

KirkMunro commented 4 years ago

That ship has sailed long ago, and I wouldn't be surprised to see variables ending with ? in use in scripts today.

At this point if we simply prioritize parsing ? as part of an operator rather than as part of a variable name when it comes to these new operators, folks who use variable names ending in ? would need to use {} or spaces (where spaces are allowed) to use those variables with these operators.

vexx32 commented 4 years ago

It's strange to hear the phrase "that ship has sailed" in this context. This is a major version change. It's not unreasonable for this to change here, I think.

Halkcyon commented 4 years ago

@rkeithhill I've used it in personal stuff, but thought it would be unclear to collaborative work since it's such an anti-intuitive thing to programmers to have symbols as part of variable names (similar to using emojis as variables)

@KirkMunro having "prioritizied parsing" sounds like an open door for bugs.

KirkMunro commented 4 years ago

@vexx32: It's not unreasonable for a breaking change since this is a major version change; however, the bar for such breaking changes should remain very high, and I don't think this comes close to passing that bar, since users could use variables ending in ? just fine as long as they use the {} enclosures to identify the variable name.

Note that you can have a property name that ends in ? as well. Currently if you try to view such a property in PowerShell without wrapping the name in quotes, you'll get a parser error.

For example:

PS C:\> $o = [pscustomobject]@{
    'DoesItBlend?' = $true
}
PS C:\> $o.DoesItBlend?
At line:1 char:15
+ $o.DoesItBlend?
+               ~
Unexpected token '?' in expression or statement.
+ CategoryInfo          : ParserError: (:) [], ParentContainsErrorRecordException
+ FullyQualifiedErrorId : UnexpectedToken

PS C:\> $o.'DoesItBlend?'
True

I'm a little surprised that doesn't parse today (why doesn't that parse?), but regardless, for such properties you need to enclose their name in quotes, in which case you could follow the quote with a ternary, null-coalescing, etc. operator without spaces and it would work just fine. I find this syntax very similar to ${x?}?.name, and I'm ok with the stance that you can use such variable/property names if you want, but such names may require extra syntax or spacing to work with ternary or null-* operators.

Halkcyon commented 4 years ago

@KirkMunro Nothing stops people from using variable-bounds going forward if they want esoteric variable names. I do agree with the others in that I wonder what the current usage of that behavior is currently in use.

People on PowerShell 7 are likely already enthusiasts and will be aware of the breaking changes. People who are not, are still using <=v5.1 and will continue to for a long time; likely until msft removes it from Windows 10 (never).

vexx32 commented 4 years ago

@KirkMunro Sure, but removing it from the standard permissible variable characters doesn't prevent users from just doing ${Valid?} as the variable name anyway. Since they'd have to do that with these operators regardless, I think it'd be better to just have it consistent, rather than have ? become a character that's only sometimes considered part of a variable name.

That already is going to be a breaking change, and I'd think it best to at least be consistent about it and go all the way rather than introduce another level of ambiguity. πŸ™‚

Halkcyon commented 4 years ago

@adityapatwardhan I think it would be better for the language to remove ? as a valid variable character. It would easily enable both null-soak/-coalesce and ternary operators in a familiar syntax which add a lot of ergonomics to the script authoring process.

KirkMunro commented 4 years ago

@KirkMunro having "prioritized parsing" sounds like an open door for bugs.

@TheIncorrigible1: Any code can open the door for bugs if it's not implemented properly. I'm just talking about a simple single-character lookahead to identify if PowerShell runs into a ternary or null-* operator when it parses a variable name that is not enclosed in variable bounds and encounters a ? character. That's not complicated, and doesn't open the door for more bugs than any other code change does.

People on PowerShell 7 are likely already enthusiasts and will be aware of the breaking changes. People who are not, are still using <=v5.1 and will continue to for a long time; likely until msft removes it from Windows 10 (never).

@TheIncorrigible1: What basis/evidence do you have of that statement? PowerShell 7 is in preview, so today it's only used by enthusiasts. That's a given. But beyond preview, if PowerShell 7 or later offer compelling features that companies need, while supporting the functionality they need, they'll use those versions. That is especially true if PowerShell 7 gets installed with the OS. The only point that enthusiasts comes into play is in organizations that don't have a business need for what PowerShell 7 brings to the table.

That already is going to be a breaking change, and I'd think it best to at least be consistent about it and go all the way rather than introduce another level of ambiguity. πŸ™‚

@vexx32 That's stretching it. It would be a breaking change if you had ?? in a variable name, but the likelihood of that is much more remote than having a variable whose name ends in a single ?. Other than that, how would the introduction of null-* operators while still supporting ? as a standard permissible variable character break scripts today?

In my experience breaking changes for nice-to-haves (which is what this seems to be) are by far more often than not rejected, and the discussion/arguments around them only serve to slow down the process of getting things done dramatically, to the point where things just stall or miss getting into a release, etc. The slow down is often necessary, because if you're proposing a breaking change evidence is needed to be able to assess the impact and justify such a change. It's hard to gather that evidence today. I'm just saying this because I'll choose getting features done now over arguing about a nice-to-have breaking change any day.

I never use ? in variable names, nor would I. I expect some folks do, though, because it can read very well in a script, and putting up unnecessary barriers to entry for PowerShell 7 just slows down adoption, especially when many people working with PowerShell aren't developers who are more accustomed to working through breaking changes.

Anyway, it is not my intent to slow down this process -- rather the opposite. But I've shared my thoughts and experience, so I won't comment further on whether or not we should push for a breaking change here.

vexx32 commented 4 years ago

Consider this contrived example:

$ValueIsValid? = @( $true, $false, $false, $true )

$ValueIsValid?[0]
# old behaviour? gets `$true`
# new behaviour? gets nothing, because the `?[0]` is interpreted as a null-conditional access.

This behaviour would already break with the proposed changes. I would prefer a consistent, clean break than a confusing break that needs a half-page explanation to list all the possible exceptions and when and where ? suddenly isn't valid, and where it still is.

Halkcyon commented 4 years ago

@KirkMunro if PowerShell 7 or later offer compelling features that companies need, while supporting the functionality they need, they'll use those versions. That is especially true if PowerShell 7 gets installed with the OS.

Having worked in a few Fortune 50s with some employee bases being in the hundreds of thousands, even getting away from what was default on the OS was a challenge (i.e., moving to v5.x). I have yet to see any place adopt Core; they'd rather just move to Python for cross-compatibility. Enabling Windows optional features was also a pretty seldom task.

I think companies work with what they have and the knowledge base of their employees over seeking out new technology or language versions to solve their problems. Some would be perfectly content staying on v2 forever and never touching the Win10/Server16 cmdlets that make life dramatically easier.

My point with all of this, is that these features are not in a vacuum. If you make a language more ergonomic, it will see greater adoption by people being interested in the tool to solve the problems they have faster. (See: C# and the growth in popularity with more/better language features)