global / nonlocal interference : is this a bug, a feature or a design hole ?

[0c148b03-074c-4cb7-a024-c061bf782863]

BPO	32361
Nosy	@bitdancer, @ilevkivskyi

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields: ```python assignee = None closed_at = None created_at = labels = ['interpreter-core', 'type-bug'] title = 'global / nonlocal interference : is this a bug, a feature or a design hole ?' updated_at = user = 'https://bugs.python.org/Camion' ``` bugs.python.org fields: ```python activity = actor = 'skrah' assignee = 'none' closed = False closed_date = None closer = None components = ['Interpreter Core'] creation = creator = 'Camion' dependencies = [] files = [] hgrepos = [] issue_num = 32361 keywords = [] message_count = 21.0 messages = ['308537', '308570', '308580', '308610', '308611', '308623', '308641', '308642', '308644', '308679', '308680', '308682', '308683', '308695', '308701', '308703', '308722', '308730', '308731', '308743', '308745'] nosy_count = 3.0 nosy_names = ['r.david.murray', 'levkivskyi', 'Camion'] pr_nums = [] priority = 'normal' resolution = None stage = None status = 'open' superseder = None type = 'behavior' url = 'https://bugs.python.org/issue32361' versions = ['Python 3.6'] ```

[0c148b03-074c-4cb7-a024-c061bf782863]

Hello,

"PEP-3104 -- Access to Names in Outer Scopes" introduced the keywords "global" and "nonlocal". but didn't make clear (to me) if this behaviour is a bug, an intentional feature, or a design hole which might be considered good or bad.

I have observed that when the nonlocal keyword gives acces to a grand parent function's variable, the presence in the parent function, of an access to a global variable with the same name, blocks it with a syntax error (SyntaxError: no binding for nonlocal 'a' found).

a = "a : global"
def f():
    a = "a : local to f"
    def g():
#        global a     # uncommenting this line causes a syntax error.
#        a = a+", modified in g"
        def h():
            nonlocal a
            a = a+", modified in h"
        h()
        print (f"in g : a = '{a}'")
    g()
    print (f"in f : a = '{a}'")
f()
print (f"glogal : a = '{a}'")

[0c148b03-074c-4cb7-a024-c061bf782863]

I forgot to mention, that I wonder if the interpreter shouldn't instead,

• either consider that since global a is used is g, it should be considered as also local to g from h point of view,
• or if h should be able to access f's version of a.
• (or if on the contrary, this ambiguity is the very reason to cause a syntax error, -- but in this case, one might need to change the text of the error message to make it more accurate. --)

[bitdancer]

What is happening here is that what nonlocal does is give the function containing the nonlocal statement access to the "cell" in the parent function that holds the local variable of that function. When you use a global statement, the name instead refers to a global variable, and there is no local cell created for that variable name. Thus you get "no binding [cell] for nonlocal 'a' found". That is literally correct from the compiler's point of view.

I agree that this message is mysterious unless you understand how python scoping works at a fairly detailed level, and thus it would be nice if the error message could be improved. The code generating the message may not know that there's a global statement for that name in effect, though, so it might not be trivial to improve it.

Would it be possible for nonlocal to effectively "chain", and turn the references in the inner function into global references? In theory the answer would be yes, but in practice it might be distinctly non-trivial, and would probably be a PEP level change to the language. It is currently documented that nonlocal does not look in to the global namespace (https://docs.python.org/3/reference/simple_stmts.html#nonlocal): "The nonlocal statement causes the listed identifiers to refer to previously bound variables in the nearest enclosing scope excluding globals.)

[0c148b03-074c-4cb7-a024-c061bf782863]

Well, David, I'm convinced this behavior is "logical" and is not some "logic" flaw. My question is more about the fact that it is desirable and was intentionally designed that way,or if on the contrary no one thought it might happen that way and it's not what was wished.

I understand that it might be a problem to link to a global (global a), a variable (h's a) which was declared in a non-global way, but is makes sense to have the presence of "global a" in g, block all possibility for h, to access it's grand parent's a.

In all case, I believe this should at lease be clearly documented in (or in relation with) PEP-3104 to make sure it has been decided that way, or else it will look like a design hole.

[0c148b03-074c-4cb7-a024-c061bf782863]

Oops, in my previous post please read :

"but does it makes sense to have the presence of "global a" in g, block all possibility for h, to access it's grand parent's a ?"

instead of

"but is makes sense to [...] grand parent's a."

[5531d0d8-2a9c-46ba-8b8b-ef76132a492c]

Does it makes sense to have the presence of "global a" in g, block all possibility for h, to access it's grand parent's a ?

From the perspective of ML-style languages with pure lexical scoping, no, it does not make sense.

But Python started with C-like simple name spaces, then nested functions and 'nonlocal' were added.

I think the answer (as usual) is that people may rely on the established convention and that it is no problem in practice.

[bitdancer]

Right, it was indeed "designed that way" in the sense that nolocal was only ever intended to access variables from the surrounding local scope, *not* the global scope. If you put a variable name in the global scope, nonlocal was not intended to be able to access it (it is then a global variable, not a local variable).

So the only question that keeps this issue open is can the error message be improved for the case where a global declaration affects the variable name in question (the message is clear when there is no variable with that name in the outer function at all).

Any change to this design would be an enhancement request and discussion of it should start on the python-ideas mailing list.

[bitdancer]

When I said "the only thing keeping this issue open" is the message, I should acknowledge that you mentioned clarifying the documentation, but as I pointed out the documentation is already clear: it says nonlocal does not access variables in the global scope, and in your example 'a' is a variable in the global scope, because it is declared to be one. We generally don't update PEPs after they are accepted and implemented; after that point the documentation is the real reference since the implementation may actually be different in detail than the PEP due to later enhancements. The PEP is still useful as an historical document. (There are exceptions to that, but that is the general rule.)

[5531d0d8-2a9c-46ba-8b8b-ef76132a492c]

I guess I'd vote for closing this, because the first Google result for "no binding for nonlocal" on Stackoverflow is quite clear.

The ideal message would be "'a' cannot be both global and nonlocal", but it would probably complicate the compiler ever so slightly.

[0c148b03-074c-4cb7-a024-c061bf782863]

David and Stefan, you're both missing my main point which is the fact that the presence of the global declaration in the parent (g) **blocks the access to the grand parent context**, which would be accessible without this global declaration (of another variable since that one is global. It just happen to have the same name) in g; and the stackoverflow post also ignores this question.

I do not disagree that this might be a desired feature (we might wish to reject this because of the potential confusion caused by this kind of name collision situation), but without any clear specification on it (since this point doesn't seem to have been discussed in the (or any?) PEP), it could always be challenged as a design flaw.

[bitdancer]

Hmm. I suppose that could be clarified in the docs. I would find it very counter-intuitive for the grandparent 'a' to be accessible, which is probably why I did not consider that an issue.

[0c148b03-074c-4cb7-a024-c061bf782863]

My interrogation is about the fact that this doesn't seem to have been a collective decision and I'm not even sure it WAS anyone decision : it might as well have been an unintentional consequence of an implementation choice.

So, in the "weakest" configuration, avoiding the implementation challenge for "design flaw" would require a language specification to at least mention that in this case, the behavior in undefined.

[5531d0d8-2a9c-46ba-8b8b-ef76132a492c]

Okay, I have never used something like that. Personally, I'd disallow the global statement in g() if 'a' is local in f().

>>> a = 10
>>> def f():
...     a = 20
...     def g():
...         global a
...         print(a)
...     g()
... 
>>> 
>>> f()
10

[ilevkivskyi]

Stefan, your last example is formally speaking OK, if one reads the "Execution model" literally. The original example is however too ambiguous, so it is good that it triggers an error.

I think there is a chance to improve the error message here, but I didn't think about this carefully (in particular, a situation where global appears after the inner function h definition worries me).

[0c148b03-074c-4cb7-a024-c061bf782863]

Ivan,

I believe, this sentence : "(the scope in which a new binding should be created cannot be determined unambiguously)" in https://docs.python.org/fr/3/reference/simple_stmts.html#nonlocal, is the one which should be clarified first, and then this would probably give the bases to improve the error message. It raises the following concerns : 1/ Is it me or there is a nasty double negation in this sentence which make it tell the contrary of what it should ? 2/ What are the conditions which would make such an ambiguity appear ? I wonder what the persons who wrote this had on their mind when they wrote it, but in regard with code reading/reviewing, my original example might be an example of such an ambiguous situation. Now the question is: are there others and what would they look like ?

[0c148b03-074c-4cb7-a024-c061bf782863]

What I mean is that we need to clarify (by giving examples ?) what make the scope for the new binding "not be unambiguously decidable", because, for an example : My example is totaly unambiguously decidable for an interpreter. It's for a human reader that it might lead to mistakes.

[5531d0d8-2a9c-46ba-8b8b-ef76132a492c]

Stefan, your last example is formally speaking OK, if one reads the "Execution model" literally. The original example is however too +ambiguous, so it is good that it triggers an error.

OK yes -- desirable, no. IMO execution models are not a great definition for scoping.

It certainly prevents Python scopes from being called "lexical scopes".

[0c148b03-074c-4cb7-a024-c061bf782863]

Stefan, You wrote : « It certainly prevents Python scopes from being called "lexical scopes". »

I don't understand why... Could you explain ?

[5531d0d8-2a9c-46ba-8b8b-ef76132a492c]

Because in the example in msg308683 true lexical scoping should, when processing g()'s name space, search for the name 'a' in the lexically closest scope f(), find it there and conclude that 'a' cannot be global.

[0c148b03-074c-4cb7-a024-c061bf782863]

I don't think there anything in the definition of "lexical scoping", which forbids to have a way to access globals from some place. Lexical scoping just means that scoping is defined in regards of the source code, by opposition dynamic scoping, which defines the scope in regard with execution.

[5531d0d8-2a9c-46ba-8b8b-ef76132a492c]

You can access globals, but not through another nested scope where the global name is shadowed.

I have to excuse myself from this discussion. It's interesting, but I don't have enough bandwidth.