Problems when running code with locals in the debugger

[OverLordGoldDragon]

demo.mp4. Only way out I found is a kernel restart. Latest Spyder via conda install.

[ccordoba12]

Hey @OverLordGoldDragon, thanks for reporting. There shouldn't be any need to run interact in our debugger because it already provides an interactive execution environment.

Therefore, I think we should disable that command and show a message mentioning that instead. @impact27, what do you think?

[OverLordGoldDragon]

@ccordoba12 Here's the problem without interact:

[impact27]

That is strange. It looks like something is resetting the local namespace. I think an old version of IPython was doing that. Could you try to update all your dependencies?

[ccordoba12]

I can't reproduce this error on Linux.

I think an old version of IPython was doing that. Could you try to update all your dependencies?

@impact27, should we increase our minimal required IPython version for 4.2?

[OverLordGoldDragon]

@impact27 conda install ipython to latest IPython 7.18.1; didn't solve either case. Any other dependencies you have in mind?

[impact27]

My guess is that some code call frame.f_locals which resets the local namespace. Because I can't replicate the bug it is very difficult for me to guess which bit

[impact27]

Maybe it is an issue with ipykernel. Could you try to update it?

[OverLordGoldDragon]

@impact27 That solved the plt being undefined within def when imported outside of it, but Clear console still kills interact. Though, for me, the problem's solved; I only used interact to counter the plt problem.

[impact27]

@impact27, should we increase our minimal required IPython version for 4.2?

@ccordoba12 We should update the minimum version of ipykernel

[ccordoba12]

Ok, to what version?

[OverLordGoldDragon]

@ccordoba12 I updated to 5.3.4 (latest), from I think 5.3.2; conda may have updated some other stuff along, don't recall.

[OverLordGoldDragon]

I just learned interact lets me a and s and everything else without !, so guess I'll be using it lot more now - hope for an option to "invert" controls in Spyder 5.x.

[OverLordGoldDragon]

I can't Arrow Up to inject earlier code? Bummer

[impact27]

I just learned interact lets me a and s and everything else without !, so guess I'll be using it lot more now - hope for an option to "invert" controls in Spyder 5.x.

There will be an option for that

[impact27]

I can't Arrow Up to inject earlier code? Bummer

Well yes that is what the history is for. It just skips single letters

[ccordoba12]

But does history work in interact?

[OverLordGoldDragon]

Uh, the plt is not defined is back, and I haven't changed a thing in the conda virtual environment (in fact I never even closed Spyder since the problem went away). And, interact history doesn't work for me at all, whether I leave input empty or type part of history.

[ccordoba12]

@OverLordGoldDragon, I was surprised when you mentioned that this problem was fixed. I really think it's a duplicate of issue #13909 (even if that issue's title seems unrelated) because we were not using the right namespace when evaluating things on the debugger.

So I think we should keep this one open until 4.2 is released and you can test it with it.

[ccordoba12]

I forgot to mention that the fix to that issue consisted on using the same namespace as interact. So if those function declarations are working for you %100 of the time on interact, they should work on the ipdb prompt in 4.2.

[impact27]

But does history work in interact?

I don't think it would, we should just disable interact in my opinion. (And fix the locals bug)

For reference: https://github.com/ipython/ipython/pull/12445

[impact27]

@OverLordGoldDragon, I was surprised when you mentioned that this problem was fixed. I really think it's a duplicate of issue #13909 (even if that issue's title seems unrelated) because we were not using the right namespace when evaluating things on the debugger.

So I think we should keep this one open until 4.2 is released and you can test it with it.

I don't think so, issue #13909 is related to a python bug with list comprehensions, and there is no list comprehension here.

[impact27]

Could you recreate this:

Python 3.8.6 (default, Oct  8 2020, 14:06:32) 
Type "copyright", "credits" or "license" for more information.

IPython 7.18.1 -- An enhanced Interactive Python.

In [1]: %debug print
NOTE: Enter 'c' at the ipdb>  prompt to continue execution.
> <string>(1)<module>()

ipdb> k = get_ipython().kernel

ipdb> loc = k._pdb_frame.f_locals

ipdb> test
*** NameError: name 'test' is not defined

ipdb> loc["test"] = 1

ipdb> test
1

ipdb> k._pdb_obj.curframe_locals["test"]
1

ipdb> loc = k._pdb_frame.f_locals

ipdb> test
1

ipdb> 

I don't understand why calling .f_locals doesn't reset the local namespace on my machine. See https://github.com/python/cpython/blob/022bc7572f061e1d1132a4db9d085b29707701e7/Lib/pdb.py#L218

        # The f_locals dictionary is updated from the actual frame
        # locals whenever the .f_locals accessor is called, so we
        # cache it here to ensure that modifications are not overwritten.

[ccordoba12]

Could you recreate this:

It works exactly the same for me in the latest 4.x

[impact27]

For reference: https://www.python.org/dev/peps/pep-0558 https://bugs.python.org/issue9633

[impact27]

So the problem with f_locals doesn't explain this behaviour because it only affects already-defined variables. Issue #13909 relates to list comprehensions but the fix could fix this as well.

[impact27]

So you are correct that this relates to Issue #13909:

In [2]: exec(compile('def a():\n    print(my_var)\na()', '<stdin>', 'exec'), globals(), {"my_var": 0})
Traceback (most recent call last):

  File "<ipython-input-2-c87500ac0845>", line 1, in <module>
    exec(compile('def a():\n    print(my_var)\na()', '<stdin>', 'exec'), globals(), {"my_var": 0})

  File "<stdin>", line 3, in <module>

  File "<stdin>", line 2, in a

NameError: name 'my_var' is not defined

[ccordoba12]

There's a fix for the Python bug you referenced above in this PR https://github.com/python/cpython/pull/11041.

It seems simple enough to add it to our debugger, at least until pep 558 is implemented.

[OverLordGoldDragon]

Another thing interact handles, for the record. (Regular for-loop is fine, but comprehension seems broken)

[ccordoba12]

Regular for-loop is fine, but comprehension seems broken

That's a duplicate of issue #13909 and fixed already by @impact27. It'll be part of 4.2.0.