Name Lookup for Wildcards

[gafter]

We are considering a number of small changes to deconstruction, out variables, and pattern-matching to allow the use of wildcards where one could declare a variable, but the user has no intention of using the value captured in that variable. Here we discuss an open issue around name lookup, and propose a resolution.

The summary of this proposal is that the use of a wildcard has no effect on name lookup elsewhere in the code.

Original (LDM) approach

The original approach described by LDM was that the lookup of the identifier _ would treat it as a wildcard if nothing were found:

class Sample
{
    void M()
    {
        M2(out _); // ok wildcard
    }
}

multiple declarations of the identifier _ in the same scope would cause _ to be considered a wildcard:

class Sample
{
    void M()
    {
        int _ = e1;
        int _ = e2;
        M(out _); // ok, wildcard
    }
}

Existing cases, where the identifier is declared once, would still introduce a variable named _:

class Sample
{
    void M(object o)
    {
        if (o is int _)
        {
            Console.WriteLine(_); //, uses declared pattern variable _
        }
    }
}

Criticism of the this approach

This approach treats an unbound identifier _ as a wildcard, a single definition of _ as causing _ to bind to that definition, and multiple definitions again treating the use of _ as a wildcard. It requires careful definition of what "multiple definitions" are. Would a declaration of _ in an outer scope, and redeclaration in an inner scope cause a use to be a wildcard?

class Sample
{
    void M(object o)
    {
        if (o is int _)
        {
            M(out int _); // allowed? wildcard?
        }
    }
}

There are corresponding implementation difficulties that may make this approach unattractive.

Proposed alternative

The proposed alternative is that

• the declaration of an identifier _ in the following contexts would always be treated as a wildcard, and would have no effect on name lookup elsewhere:
  • an out argument,
  • the left-hand-side of a deconstruction, or
  • a pattern variable
• the use of the simple identifier _ as an expression as an out argument, or as a variable in the left-hand-side of a deconstruction, would be bound to a variable of that name. If no variable of that name is found, it is treated as a wildcard. This is similar to the treatment of var as a type.

class Sample
{
    void M(object o)
    {
        if (o is int _) // wildcard by 1.
        {
            M(out int _); // wildcard by 1.
        }
        (int x, int _) = e; // wildcard by 1.

        M(out _); // wildcard by 2.
        (o, _) = e; // wildcard by 2.
        (int x, _) = e; // wildcard by 2.

        Console.WriteLine(_); // error: no _ in scope
    }
}

This is much simpler to implement and I believe much easier to explain.

[gafter]

/cc @dotnet/ldm @jcouv @AlekseyTs @jaredpar @VSadov

[HaloFour]

To clarify, if there are no ambiguous overloads of a method declaring out parameters you may use _ as a simple wildcard without the declaration syntax?

//given
void M(out int x) { x = 123; }

// all are equivalent statements?  (assuming no variable _ in scope)
M(out var _);
M(out int _);
M(out _);

If no variable of that name is found, it is treated as a wildcard.

While I do think that this is an improvement on the original proposal I still there that there is too much room for ambiguity and the potential for accidentally overwriting legally declared fields/variables:

void Foo() {
    M(out _); // intended to throw away result
}

private int _; // oops, just added this field and changed the meaning of my program!

I'd argue that for out parameters you could only use a wildcard with the declaration:

void Foo() {
    M(out var _); // no ambiguity, definitely a wildcard
}

As for deconstruction, I'd argue that the use of _ without a declaration should always be considered a compiler error if there is a field/local with the name _ in scope:

private int _;

void Foo() {
    {
        int o, _;
        (o, _) = e; // compiler error, cannot deconstruct into _
        (o, var _) = e; // no ambiguity, definitely a wildcard
    }
    {
        int o;
        (o, _) = e; // no _ in scope, definitely a wildcard
    }
}

I acknowledge that this is a hard-line approach, but in my opinion it's worth being a little more obnoxiously stricter in order to avoid the potential that any legit variable/field is overwritten. For people who don't use the _ identifier there would effectively be no difference.

[vbcodec]

@HaloFour Forcing to deconstructions like (x, var _, var _, var _) and inability to deconstruct into _ is antithesis of improvement.

For private / protected / external _ there may be warning (where using _) for such cases.

@gafter Disabling leakage var _ from deconstruction into enclosing scope is further complication already complicated rules of scoping described #12939

Do we need wildcards in typeswitch at all ? imo, it is useless here

Can you explain why characters #, @, $, % can't be used for wildcards ?

While presented alternative approach is simpler, I would stick with previous proposal which is bit more complex, but more aligned with rules for other variables.

[gafter]

Do we need wildcards in typeswitch at all ? imo, it is useless here

Providing the user a way to avoid declaring an identifier allows us to warn when the declared identifier is unused.

Can you explain why characters #, @, $, % can't be used for wildcards ?

We don't think any of those are an improvement over _, and most other languages with a similar concept appear to agree. A wildcard, representing as it does something that the user intends to ignore, should be as low profile as possible, and nothing is lower profile than the Unicode LOW LINE. Users already declare _ when they intend a variable to be ignored, and this choice recognizes and blesses user preference.

[dsaf]

Shouldn't that 0.01% of developers who are currently actually using _'s value be somehow punished anyway? I am more worried about general leaking to outer scope #14697.

@vbcodec

Do we need wildcards in typeswitch at all ? imo, it is useless here

Yeah, not sure why we need two ways:

if (o is int _) {}

if (o is int) {}

[gafter]

Do we need wildcards in typeswitch at all ? imo, it is useless here

Yeah, not sure why we need two ways:

if (o is int _) {}

if (o is int) {}

That isn't a typeswitch.

In the context of an is expression, however, this may disambiguate some situations, thereby enabling the use of tuple types:

if (o is (int x, int y) _) {} // test for the type ValueTuple<int, int>; nothing placed in scope
if (o is (int x, int y) {}  // test for any tuple whose value contains two integers; places x and y in scope

[dsaf]

It makes sense in such situations I guess. Although the names are probably not relevant in first case:

if (o is (int, int) _) {} 

or

if (o is (int _, int _) _) {} 

[dsaf]

Also why not this?

if (o is (int, int)) {} 

[HaloFour]

@vbcodec

Forcing to deconstructions like (x, var _, var _, var _) and inability to deconstruct into _ is antithesis of improvement.

Hey, that's my word! 😁

The only thing worse than a slightly more verbose improvement is a syntax that can accidentally lead the developer into a subtle runtime bug. Allowing deconstruction into an existing _ is a big potential pit of failure.

Note that this would apply specifically to deconstructing into existing variables which I expect will be a minority case. For people using declaration deconstruction syntax you'd only need var (x, _, _, _). I doubt most people would even notice.

For private _ there may be warning for such cases.

It's my opinion that a warning wouldn't be sufficient. You could add a protected field named _ in some base class and have no idea what code you impact further down the pipeline, including in other projects.

@gafter

We don't think any of those are an improvement over _,

I tend to agree. The closest I might suggest is . but I imagine that comes with its own baggage of parsing issues. I still favor * over all, but I understand the type ambiguity that introduces (and I'm jealous of VB.NET for the first time in a very long time).

@dsaf

Shouldn't that 0.01% of developers who are currently actually using _'s value be somehow punished anyway?

You know that the most active "abusers" would be those Fortune 50s with massive code bases that have every other MS VP on speed dial and would call them up screaming if they even suspected that their builds might break. There are some threads on the coreclr repo sharing some interesting stories of just this sort of thing happening with updates to .NET impacting internals that nobody should have ever relied on anyway, updates that had to be rolled back.

Anyhow, MS made _ a legal identifier, why should people who took advantage of it be penalized?

[dsaf]

@HaloFour

You know that the most active "abusers" would be those Fortune 50s with massive code bases that have every other MS VP on speed dial and would call them up screaming if they even suspected that their builds might break. There are some threads on the coreclr repo sharing some interesting stories of just this sort of thing happening with updates to .NET impacting internals that nobody should have ever relied on anyway, updates that had to be rolled back.

And they are rushing to use C# vNext? And they will scream you say? Did it help e.g. save Silverlight? Based on what Microsoft is generally doing, enterprise is irrelevant. Unless it uses Node.js with Git and hosts it on Azure.

[vbcodec]

@gafter

We don't think any of those are an improvement over _

But these characters do not conflict with any name, and there won't be needed any tricks to avoid ambiguities. Doesn't this outweigh 'low profile' of _ ?

[HaloFour]

@dsaf

And they are rushing to use C# vNext?

This is also why the team has refused to add any new warnings to the compiler for existing code. Can't risk breaking some build that happens to be riddled with awful code.

https://github.com/dotnet/corefx/issues/1420#issuecomment-96430564

We have had to revert refactorings because customer applications were taking a dependency on the method names in various stacks. Ugh.

Who knows, maybe Microsoft could stand to be significantly more abusive towards their customer base. It seems to work for Apple.

[dsaf]

@HaloFour more abusive than the whole Sinofski fiasco? https://blog.ailon.org/how-one-announcement-destroyed-the-net-ecosystem-on-windows-19fb2ad1aa39#.2b9c211pz Renaming a method is kind of easier than switching a GUI framework.

[CyrusNajmabadi]

Let's keep the conversation on topic. Thanks :)

Doesn't this outweigh 'low profile' of _ ?

Currently we don't think so. "_" as a wildcard fits into how people are generally using that identifier already today. So we think it's nice to be able to just codify that concept, even if it means we do need to put in the due diligence to make sure we don't break any existing code.

[dsaf]

@CyrusNajmabadi sorry, I'll stop now :).

[DavidArno]

Regarding the new proposals, I think @HaloFour has nicely summarised what I also see as a better solution than @gafter is proposing. I too think we need very strict - _ variable in scope or _ is a wildcard - dichotomy solution here to minimise the chance of really hard to find bugs.

There is one use-case that's missing from both the OP and @HaloFour's post. I think is important to clarify that following extra wildcard case should also be supported:

_ = e; // wildcard, but not explicitly covered by @gafter's 2nd case

[bondsbw]

How about using ~ (tilde)?

if (o is int ~)
{
    M(out int ~);
}
(int x, int ~) = e;

M(out ~);
(o, ~) = e;
(int x, ~) = e;

It is almost as low profile as _, and has an appearance that conveys the ephemeral nature of wildcards.

[DavidArno]

@bondsbw,

~ is the bitwise complement operator. Just like -, #, * etc, it therefore already has a meaning. _ is already used by many of us as a pseudo wildcard, so already has that meaning in the minds of 99% (pure guess) of those that use it.

[bondsbw]

What makes ~ better, in my opinion, is that it cannot be used in similar circumstances. Nobody is going to confuse bitwise complement with wildcard.

And to echo @HaloFour's point, it doesn't suddenly change meaning when you are working elsewhere.

[bbarry]

If you were considering other symbols, I'd suggest @. It is already a valid character at the start of an identifier (though it doesn't affect the name of the identifier). Reading if (o is int @) as if it were valid c# without a wildcard appears to be a pattern into a variable with no name.

That said, I like _ as a wildcard even if it is ambiguous with existing code. I'd be happy with the rule:

1. Any pattern or deconstruction syntax involving a wildcard where there is a variable by the name _ in scope is an error.

[CyrusNajmabadi]

as mentioned already, we are trying to avoid using other symbols. There's no end to the number of syntacticly unambiguous options we have if we go with something else. But that's not the goal here. The goals are:

1. to use the character today that people already routinely use to mean 'i don't care about this variable'.
2. to use a variable today that has a familiar meaning for people using other languages that supports this concept.

It's similar to the issue we had when we introduced generics into the language. We could have gone with different syntax than <>. It would have helped us avoid ambiguities in the language. However, we felt that there was enough merit in those characters (esp. with how parametric polymorphism occurs in other languages) to use them, even though they were more difficult to fit into our language.

To that end specified above, '_' is an ideal wildcard character. It just needs to be added to the language in a careful manner as we view back compat as a very high bar.

[bondsbw]

Then I would prefer to change rule 2 so that _ is always treated as a wildcard in those circumstances.

@_ would still be allowed if you want the identifier.

[HaloFour]

@CyrusNajmabadi

I know that I can be a big pain in the butt but all I'm trying to do is help in finding that point where it's safe to determine that _ would be a wildcard while trying to mitigate scenarios where the compiler/developer might do the wrong thing. I know that my code won't be affected as I've never resorted to using _ as an identifier. But I know people who have, both in scenarios where they wished to ignore the result and in cases where they wanted a simple syntax for selecting a property. I don't think I've seen it used elsewhere, but it wouldn't surprise me if it has.

Which leads me to ask what the expected/wanted behavior would be in the following scenario:

var (x, _) = e;
var query = employees.OrderBy(_ => _.FirstName);

This change might also piss off the fledgeling efforts to port Underscore to .NET. 😁

[gafter]

@HaloFour

var (x, _) = e;
var query = employees.OrderBy(_ => _.FirstName);

That code is perfectly fine. The first _ is a wildcard, and the second one means the same as it always did.

[DavidArno]

@HaloFour,

var (x, _) = e;
var query = employees.OrderBy(_ => _.FirstName);

Even by the stricter "no wildcard use if an _ var is in scope" rules, that code would be fine as the lambda parameter isn't in scope when the wildcard is used in the deconstruction.

I know that my code won't be affected as I've never resorted to using _ as an identifier. But I know people who have, both in scenarios where they wished to ignore the result and in cases where they wanted a simple syntax for selecting a property

I have used it, quite often as a pseudo-wildcard in lambdas. I don't get using it for "simple syntax for selecting a property" though: x is a far more established convention there. I would far prefer the team just risked breaking my code and made _ a wildcard only (requiring @_ for existing identifiers). The - at most - few hours of rewriting would be well worth avoiding making the language more complex. But that's (one of the many reasons) why @MadsTorgersen is running the language team; not me :smile:

[HaloFour]

@DavidArno

x is a far more established convention there.

Or any number of single-letter identifiers. But it was never wrong to use an underscore.

I would far prefer the team just risked breaking my code and made _ a wildcard only (requiring @_ for existing identifiers). The - at most - few hours of rewriting would be well worth avoiding making the language more complex.

To be honest, so would I. Rip the Band-aid and eliminate all possibility of ambiguity. I'm not even sure that would be possible, though. Code like var _ = 123; and M(out _) would suddenly take on a completely new meaning. Of course if you then never used that variable the difference probably wouldn't really matter.

I'd be all game for changing the rules of legal identifiers to require an _ to be followed by an alphanumeric character.

[iam3yal]

Maybe we need to add a strict option for C#. haha.. j/k

I like the underscore approach a lot aka time to disallow underscores completely, well at least single ones.

@gafter What happens when you use more than a single underscore character? is it still a wildcard? honest question. 😆

void M()
{
    M2(out __); // Allowed?
}

[alrz]

If we ever want "sequence wildcards" in array/tuple patterns, I think that would be the syntax:

tuple is (1, __) // tuple is (1, _, _, _)
arr is { 1, 2, __ } // matches an array with length > 2 
arr is { 1, 2, ___ } // matches an array with length >= 2
arr is { 1, 2 } // matches array with length = 2

Rel. #5811, #10631

[DavidArno]

@alrz,

Whilst *** might have been OK for sequence patterns, it really doesn't work with underscores as many fonts join them together, making __ and ___ far too hard to tell apart.

[HaloFour]

@alrz

@DavidArno just beat me to it. The _ character really does not lend itself to being repeated as visually they would just blend together. Maybe ... could be a "sequence wildcard", although then we'd have two completely different wildcards.

[alrz]

@HaloFour @DavidArno This is the exact syntax that used in Mathematica for the same purpose. However, I think a single "zero or more" wildcard is sufficient as you can simulate the other one with help of a regular wildcard: arr is { 1, 2, ___ } vs arr is { 1, 2, _, ___ }.

[RedwoodForest]

One thing that disappoints me about this approach (if I understand it correctly) is that it doesn't allow using wildcards for parameters to lambda expressions. Sometimes callbacks include parameters you don't have a use for and it would be nice to have a way to ignore them.

For example when you only have a self-signed certificate in your development environment and you want to disable checking the certificate in development:

ServicePointManager.ServerCertificateValidationCallback = (_, _, _, _) => true;

Or if you wanted to do partial validation you might only need the certificate and policy errors but not the sender or chain:

ServicePointManager.ServerCertificateValidationCallback = (_, certificate, _, sslPolicyErrors) => {...};

I understand the new approach is more straightforward (and it may be worth going with it for that reason) but this was the use case that immediately came to my mind when wildcards were proposed in the first place so it's too bad it won't be supported.

[DavidArno]

@lawrencejohnston,

If you take a look at #14794, @grafter does give an example of using _ as a wildcard parameter in a lambda. Whether it will happen is, like everything else in the discussion, not yet decided and subject to change.

[RedwoodForest]

@DavidArno

Right, the original proposal included it but my understanding is that this alternate proposal would not support them. By "One thing that disappoints me about this approach" I was referring to the alternate approach proposed in this issue.

[iam3yal]

@alrz, @HaloFour, @DavidArno Do you guys know whether repeated underscores are going to be considered a wildcard? I mean I'd assume it won't but dunno based on other features like digit separators it seems a reasonable question where it's perfectly valid there.

[DavidArno]

@eyalsk,

Absolutely no idea and I'm beginning to think that the team don't know either yet. I think we are seeing the start of C# being genuinely "developed in the open" in that these are just ideas @gafter is throwing around in the open to gauge community reaction before taking it to a design meeting for discussion. So anything could happen, I guess (or I'm being naive :grin:)

[JesperTreetop]

For those of us who missed it, what's the reason * can't be used here? * is more universally understood as a catch-all and a wildcard, it won't change the meaning of previously legal programs, and you can't define a _ variable in scope and change the meaning at a distance of the wildcard. (Far more likely than, say, defining a var class...) There's precedence of otherwise valid identifiers being keywords in C#, but I can't think of one that isn't a word or at least alphanumeric.

[HaloFour]

@JesperTreetop

That int * is a valid type in C#, although pointers can't be used in is expressions nor type-switch expressions, nor could it be used without an identifier in out declarations.

[CyrusNajmabadi]

@JesperTreetop From above:

The goals are:

1. to use the character today that people already routinely use to mean 'i don't care about this variable'.
2. to use a variable today that has a familiar meaning for people using other languages that supports this concept.

[JesperTreetop]

@HaloFour

Right, gotcha, missed this.

@CyrusNajmabadi

Still don't think, no matter what some people use _ for, that the C# team should codify an intent now that we're seven versions in with many millions of lines of source code. Overloading possibly valid identifiers seems dicey because it makes people possibly have to carry two meanings around, the exception being if it had been ignore or skip where you could (to my mind) make a much stronger argument that the intended use overlaps completely with any ad-hoc convention. It would still have the problem where if someone defined something in scope with that name, the behavior would "magically" change. (And as with anything introduced after C# 1.0, you couldn't make those words keywords, only contextual keywords, because you'd break existing programs.) So I'd still prefer something outside of the overlap with possibly valid identifiers.

[CyrusNajmabadi]

I personally disagree. I very much dislike it when languages go out of their way to avoid these problems, only to end up with an end result that feels inconsistent and unpleasant to use. I'd rather codify patterns and practices and move forward with a healthier feeling set of code.

In practice these issues just have not caused problems for us in the past. I'd prefer for us to not bend over to obsessive paranoia, and just be cognizant of how code is actually written an what the end impact will be.

In this case, the impact seems extremely tiny. If you're using today as a wildcard, you'll continue being able to do so. if you're using as an actual variable, you'll still be able to do so. And, if you end up adding a wildcard that conflicts with that you'll know immediately because your code won't compile anymore.

I think that's fine. Back compat is ensured. Any mistakes are found immediately. And there is a simple way to express this concept in a manner that fits with our ecosystem and the greater programming community.

[HaloFour]

@CyrusNajmabadi

Who's goal is that, exactly? Again, seems like a case of the destination being determined long before the journey has been plotted.

And, if you end up adding a wildcard that conflicts with that you'll know immediately because your code won't compile anymore.

This is patently not the case. I've provided numerous examples where even the stricter rules above produce terribly ambiguous situations that can lead to very subtle overwrite bugs. Simply put, said situations are impossible to avoid, not without further serious restrictions to the applicability of wildcards.

[CyrusNajmabadi]

This is patently not the case. I've provided numerous examples where even the stricter rules above produce terribly ambiguous situations that can lead to very subtle overwrite bugs.

I've found your cases super-non compelling. For example, the case of someone adding a field named _ in their file.

Such a situation is similar to someone adding an alias to 'var' in their file. Sure, it can happen. Sure, it can change meaning. Do i think that's a relevant case to consider? No.

We have to practically look at how our language is actually used out there. And working to avoid pathological conditions is not something we do. it's simply over-constrains langauge development and hurts the end language for most users to avoid problems that do not actually arise in practice.

I personally do not think it's healthy to design hte language in such a fashion. And while there is a contingent of people giving feedback that they would prefer that, it's not something that we honestly feel is appropriate to do.

Our experience has borne this out through numerous releases. Many (all?) interesting features have had this sort of potential 'gotcha'. And in reality its not an issue, and the language has been healthier and more enjoyable to use precisely because we did not try to ensure that such cases could never exist.

[HaloFour]

@CyrusNajmabadi

I've found your cases super-non compelling.

You could've just said, "_ you." I'll pretend that's a wildcard and not an identifier. Of course with your compiler nobody could ever really know for sure.

[CyrusNajmabadi]

@Halofour. Please keep the discussion respectful. I realize that we may make decisions you disagree with. It is not personal. It is because we simply weigh things differently. I've heard your cases that you are concerned with, and i'm simply stating that i do not find myself concerned with them.

This has come up on several threads, but i think it bears repeating: It is a non-goal when we make language changes to make everyone happy. We weigh out a lot of options every time, and it is nearly always the case that for every interesting language area there is no perfect answer that maximizes value for every consideration. We end up going with a result that we feel is the best balance of positives vs negatives. If that results in something you don't like, it's not personal. It's just how things work when you have to design for an extraordinarily large audience with many different desires.

[JesperTreetop]

Such a situation is similar to someone adding an alias to 'var' in their file. Sure, it can happen. Sure, it can change meaning.

Then, as long as we're talking about a new feature, why not take it the way it is intended - as a knock against that option and a reason to keep exploring other options too? Subtly changing the meaning of people's code in some rare circumstances could of course be the best option, but it's hard to know without anything else to compare it to, and without having had that discussion first. Preempting that discussion makes it harder; doing it in the face of provided examples to the contrary seems downright hostile.

We're just trying to be in the room too and bring the healthy critique and alternative viewpoints that are unavoidable parts of testing a good design - please don't fault us for that. It may be the case that there has been a grand discussion about this, carefully weighing every possible alternative, but it's hard to tell when only the conclusions are shared.

[CyrusNajmabadi]

No discussion has been preempted.

doing it in the face of provided examples to the contrary seems downright hostile.

Nothing was preempted there. I simply stated that i found the use case non-compelling. That's an honest and true assessment of the example that was provided. If i'm to care about a potential pitfalls i have to at least truly believe that they're really going to happen. :)

but it's hard to tell when only the conclusions are shared.

This is a proposal. :)

[CyrusNajmabadi]

why not take it the way it is intended - as a knock against that option and a reason to keep exploring other options too?

Because, not all things are equal. Literally any language change we make can have some knock against it. At a certain point though, if the knocks are not important enough, then exploring other options is not appropriate.

Furthermore, as has been mentioned in the thread, the other options also have knocks against them. There are rarely "slam dunk" cases :). Almost all the time, there's just a series of tradeoffs. In this case i've mentioned why i believe those other options are inferior to the option specified. That's not to say that there are not pros/cons to each side. Simple that under my own weighting of things, why one is better than the other.

[vbcodec]

I agree with @CyrusNajmabadi here. Irrationally shaping feature just to support careless programming do not make sense. With current (and previous) rules existing code compile flawlessly. There must be small warning anyway, if modified variable _ is defined outside function, just to make dev aware of potential problem. There are dozens such warnings already defined, and there is not any problems with them

[HaloFour]

To me ambiguity is the epitome of bad language design. I dislike the new scope rules, but at least the result is unambiguous and cannot lead to subtle bugs.

The example using a field is just a simple one and to demonstrate that said identifier doesn't even need to be declared in the same method (or even the same class), leading to the developer not even knowing that they're accidentally overwriting anything. The same situation could arise within a single method with a legally-declared variable.

This proposal is obviously a step to avoid those circumstances. It's close. It can never be perfect, but it can make a few relatively minor changes which would help to avoid those potential edge case bugs.