Unable to see plots made with Matplotlib while debugging

[spyder-bot]

From al.dan...@gmail.com on 2011-04-12T17:33:50Z

This script, when run within spyder, works fine and produces a contour plot:

!/usr/bin/env python

from matplotlib.pyplot import imshow, show import numpy as np x = np.random.rand(4,5) imshow(x) show()

However, if I enter debug mode and put a breakpoint at the imshow(x) line then manually type the imshow(x) and show() commands at the (Pdb) prompt, no plot appears. I only get a reference to the plot object:

(Pdb) imshow(x) <matplotlib.image.AxesImage object at 0x183f5150> (Pdb) show() (Pdb)

Would be a nice enhancement to be able to plot variables in debug mode.

Python 2.7.1 NumPy 1.5.1 matplotlib 1.0.1 (WXAgg backend) RHEL 4.8 x86_64

Original issue: http://code.google.com/p/spyderlib/issues/detail?id=620

[spyder-bot]

From al.dan...@gmail.com on 2011-04-12T15:40:57Z

Forgot to add that this is with spyder 2.0.8 and ipython 0.10.1. Also, the backend when spyder runs is Qt4Agg. I get WXAgg when I run ipython directly from a terminal window (that is, independently from spyder).

[spyder-bot]

From ccordoba12 on 2011-05-15T19:31:44Z

Labels: Cat-Debugger

[spyder-bot]

From ccordoba12 on 2011-08-16T11:41:02Z

issue #733 has been merged into this issue.

[spyder-bot]

From eigenjoh...@gmail.com on 2011-11-10T07:23:21Z

this fix would make this application a matlab killer

[spyder-bot]

From kavaldj...@gmail.com on 2013-05-12T21:43:03Z

Has this been fixed or addressed somehow?

I agree with comment #4 : this would make Spyder closer to a ML killer

[spyder-bot]

From contrebasse on 2013-05-13T00:37:52Z

1: this is normal, the defalut backend for matplotlib is WXAgg. You can change it in your script :

import matplotlib matplotlib.use('Qt4Agg', warn=False)

[spyder-bot]

From jed.lud...@gmail.com on 2013-05-13T07:45:45Z

Additional recent diagnostics after doing a bit more testing:

• Plotting fails while debugging inside of Spyder when the script is executed inside the current interactive Python session or interactive IPython session. This is true for Qt4Agg or WXAgg backends.
• Plotting during debugging works fine inside Spyder if the script is set to execute in its own dedicated Python interpreter. This is also independent of backend.
• Plotting during debugging works fine from a session of "ipython qtconsole" when executed in a system console.

Soyder 2.3.0dev ( revision 0bb65fdb4b6e ) Python 2.7.3 32-bit IPython 0.13.2 Matplotlib 1.2.0 Windows 7 (64-bit)

[spyder-bot]

From kavaldj...@gmail.com on 2013-05-14T13:45:31Z

8 Jed, thanks!!

Running debug in a dedicated interpreter works for me, thanks for the great hint!

I apologize if I appear greedy, but is there a simple way to make the figure non-modal?

Right now it blocks the interpreter as it is stopped in Pdb, may be there is another trick I can use? I am just calling imshow(data), show().

Thank you so much in advance! dan

[spyder-bot]

From jed.lud...@gmail.com on 2013-05-14T16:55:16Z

@-kavaldijiev: Making the figure non-modal is the difficult piece to solve. It looks like Carlos looked into solving this under issue #733 , and it may not be straightforward without some interesting thread work. It probably has something to do with matplotlib's interactive mode having to run in parallel with pdb, and you'd have to figure out how to get those threads to cooperate.

Note that if you are trying to plot during debugging in the IPython Qt Console, it blocks, too, after you issue the "show()" command. It's not completely interactive there, either.

Running in a dedicated interpreter looks like the best workaround for now.

[spyder-bot]

From kavaldj...@gmail.com on 2013-05-15T14:54:57Z

@-jed Thank you for the detailed explanation!

I will manage with modal figures. But it is definitely worth making it work in the long run, it could be extremely attractive to the MATLAB crowd.

[spyder-bot]

From jed.lud...@gmail.com on 2013-05-16T07:57:29Z

@-kavaldijiev: After a little more investigation, there is another workaround that should provide you a way to interactively plot while debugging. It seems that it's probably the Qt GUI components that are getting in the way of interactive plotting. If you run IPython without the Qt console interface, interactive plotting works while debugging without blocking. Here are the commands:

1. Open a Command Prompt and cd to the directory where your script is.
2. Run "ipython --pylab" at the prompt.
3. Inside IPython, run "%run -d .py"
4. You should now be able to step through the code using pdb commands, plot using imshow(), and the plot should draw without blocking command line input.

[spyder-bot]

From jed.lud...@gmail.com on 2013-05-16T15:11:32Z

And, after a bit more research, I found this answer: http://stackoverflow.com/questions/12822762/pylab-ion-in-python-2-matplotlib-1-1-1-and-updating-of-the-plot-while-the-pro/12826273#12826273 So, if the code in the original post is modified in this manner:

from matplotlib.pyplot import imshow, pause import numpy as np x = np.random.rand(4,5) imshow(x) pause(1)

you can get the plot to update, interactively, inside of a Spyder interactive console during debugging without blocking! So, use pause(1) instead of show(). The plot only becomes interactive for the pause duration, so if you want more time to zoom or pan in the data, use pause(10) or pause(20). Works in the IPython Qt Console, too.

[spyder-bot]

From kavaldj...@gmail.com on 2013-05-16T22:35:04Z

@-jed, thanks, both methods work!

The command prompt method appears more robust -- I can close the opened figures, and the figures can get the focus (for zoom, etc) until closed.

The pause method is obviously preferable since it's within Spyder. The pause returns the keyboard control, but the window shouldn't be closed, otherwise the interpreter hangs in. Opening more new figures is shaky -- sometimes succeeds, other times does not, but I did not spend much time testing.

In any case, any of those are extremely useful for debugging, even not as easy and robust as ML.

Cheers, dan

[spyder-bot]

From al.dan...@gmail.com on 2013-05-19T21:08:30Z

Glad to see progress after a couple of years. Yes, the pause(n) trick works well for me. Thanks, @-jed, for following through with a solution!

[spyder-bot]

From jed.lud...@gmail.com on 2013-05-20T09:13:28Z

I can't take much credit for anything other than investigation. The pause(n) function is a matplotlib feature I just happened to discover.

Ultimately, the root of this problem is really the interplay between matplotlib drawing event loops and Qt application event loops, and I'm not sure there is really a "Spyder" solution to it. It probably has to be addressed at the matplotlib level. As eveidence of this, IPython running in Qt consoles suffers these same limitations.

[spyder-bot]

From jason.bo...@gmail.com on 2014-07-07T05:33:04Z

This issue seems to have been solved in IEP: https://bitbucket.org/iep-project/iep/issue/297/interactive-plotting-is-not-working-during

[spyder-bot]

From ccordoba12 on 2014-07-07T07:53:08Z

issue #1848 has been merged into this issue.

[jandyman]

I was just wondering if someone could give me a little perspective on where we are headed with all this. I suspect that I'm not the only one out there who has spent a lot of time using Matlab and is very attracted to the Python world for various reasons and senses that true productivity is tantalyzingly close but like a mirage seems to be always just out of reach. The pause() workaround is great and the responsiveness of the Spyder team is truly impressive. And now that the slow editor problem in El Capitan is seemingly slayed in 3.0 beta, it all seems so close. And yet ...

So where does an expert think this is all headed? From what I can gather, a big part of the problem is Matplotlib itself. And I also gather that guiqwt isn't is great shape right now either. So do folks think that we are headed towards a situation where we can debug in an IDE, hit breakpoints, and do interactive plotting from the breakpoints, all somewhat seamlessly? I think that is what all ex Matlab users want, and some of us are wondering whether we should pony up the $200 for Matlab Home Edition, going back to The Borg and walking away from the Python approach and the appeal of community based software and the Python language.

It may sound like I'm complaining, but I'm actually just trying to figure out how to invest my time. I really want the Python way to be the Matlab killer it could be ...

[ccordoba12]

@jandyman, please don't despair :-) We've been doing a massive reorganization/cleanup for Spyder 3.0 that would be very beneficial for Spyder 4.0, where we plan to solve this issue.

The real problem is not with Matplotlib, it's with the IPython/Jupyter architecture that blocks the console while on debugging, preventing us to run a more featured debugging session (with history browsing and plots) as in Matlab.

But let me assure you that after 3.0 is released (in a month or so), all our efforts will be directed to solve the many outstanding and very obnoxious debugging issues Spyder has. If everything goes well and we succeed, we plan to release our fixes in 4.0 (as I said) by the end of the year, or early in the next one :-)

[ccordoba12]

I also gather that guiqwt isn't is great shape right now either

That's a @PierreRaybaut project (the same one who created Spyder). Unfortunately he doesn't have much time for open source lately :-(

[PierreRaybaut]

I also gather that guiqwt isn't is great shape right now either

That's a @PierreRaybaut project (the same one who created Spyder). Unfortunately he doesn't have much time for open source lately :-(

@ccordoba12, @jandyman: That's right, unfortunately. However, I did some significant work recently on guiqwt: I have solved the PyQwt obsolescence issue by reimplementing it in pure Python (see PythonQwt project) so that guiqwt v3 does not rely on PyQwt anymore. I don't think that guiqwt is in bad shape right now: thanks to recent developments, it works on Python 2.7, 3.0-3.5 and with Qt4-Qt5. But the project is not growing, because I don't have time for managing it (i.e. creating new features, promoting the library, etc.) -- I'm only doing maintenance, so that the current features continue to work as before. As it is, note that it's still far, far more performant than matplotlib (or MATLAB) for showing and manipulating large images in real-time (zomming/panning, adjusting contrast, plotting cross sections, etc.) for example, and it's a way ahead matplotlib regarding interactivity.

Regarding @jandyman original post, I can't think that there is no solution out there to use Python/Spyder as a good replacement for MATLAB. Personnally, in 2009, I was already able to do so and without any compromise. But, the solution to implement this in 2009 does not apply anymore to the current state of the code: at that time, Spyder was relying on a "pure" Python interpreter. As @ccordoba12 mentioned, Spyder is now relying on IPython/Jupyter: this is a great thing as it enabled a lot of new features but it also has its drawbacks. Anyway, I don't see why it could not be possible to interact the same way in Spyder than in MATLAB.

[felipeespic]

Hi everyone,

I am one of the users that has switched from MATLAB to Python. I think that Spyder is a great IDE, but the problem of no interactive plotting while debugging is an important drawback for people that comes from MATLAB.

I think it should not be so difficult to solve it. For example, I am running IPython from a system terminal (no qtConsole), and I can do interactive plot (guiqwt) flawlessly, either from IPython, or ipdb sessions. Also, I have command history while debugging. I think that if you just replace (or add the option) of running ipython from a terminal (no qtConsole), this problem will be solved.

Please, try to fix this, since I think this is the only thing that is stoping a lot of people from using Spyder.

Thanks,

Felipe

[ccordoba12]

I think it should not be so difficult to solve it.

It is. The libraries and technologies used by IPython in the terminal, and qtconsole are completely different.

Not for anything this bug has been opened for 5 years, and it still will be for some more time ;-)

[nanochap]

I need this feature because I use Spyder as the main IDE to develop image processing algorithms, viewing the images while debugging is a very common feature request.

[mmagnuski]

I would like it too, I use Spyder mainly for debugging.

[ccordoba12]

@mmagnuski, this is possible right now by using the %plot magic while debugging, like this

%plot plt.plot(range(10))

But for this to work you need to set your Matplotlib backend to Inline.

[hmaarrfk]

@tacaswell I was having a conversation with @WeatherGod and he thought you might be able to chime in.

It would be cool if this worked "seamlessly" though I understand it is hard.

For now, I use the pause(10) trick.

[tacaswell]

See https://github.com/matplotlib/matplotlib/pull/4779 for the gory details.

The very short version, for the GUI to be responsive it's event loop (which is essentially an infinite loop waiting on user I/O) needs to be running. The debug prompt is also in infinite loop waiting for the user to type. At the 'normal' prompt there is some delicate integration work done to let these two loops share (basically, the prompt lets the GUI loop run until a single key is hit, processes the keys, and then restarts the GUI event loop). At the debugging prompt this integration is not done and the figure seems 'dead'. This will affect any GUI based window.

The reason calling plt.pause() works is that explicitly runs the GUI event loop for the given number of seconds.

The inline backend works because it just produces a static png (so is always 'dead').

[ccordoba12]

Thanks for the explanation and the linked issue Thomas!

[impact27]

This PR would solve it I think: https://github.com/ipython/ipykernel/pull/438

[ccordoba12]

Great work! Thanks a lot @impact27!

[rodrigovimieiro]

Any solution for this? I can't close the figure after plt.pause(1). I tried plt.close() and pause again but n success. Thanks

[impact27]

Any solution for this? I can't close the figure after plt.pause(1). I tried plt.close() and pause again but n success. Thanks

You have to give the figure number as an argument to plt.close() (or 'all'). The solution for now is plt.pause. It will be solved when/if ipykernel merges https://github.com/ipython/ipykernel/pull/438. I personally use a modified version of ipykernel with this branch merged.

[rodrigovimieiro]

Any solution for this? I can't close the figure after plt.pause(1). I tried plt.close() and pause again but n success. Thanks

You have to give the figure number as an argument to plt.close() (or 'all'). The solution for now is plt.pause. It will be solved when/if ipykernel merges ipython/ipykernel#438. I personally use a modified version of ipykernel with this branch merged.

It didn't work. I run plt.figure(1); plt.imshow(myimage);plt.pause(1);plt.close(1) and no results. I am on IPython 7.11.1, Spyder 4.0.1 and python 3.7.5

[impact27]

I was able to reproduce. Please open a new issue about that. As a workaround you can close them with the cross while under plt.pause

[rodrigovimieiro]

I was able to reproduce. Please open a new issue about that. As a workaround you can close them with the cross while under plt.pause

This workaround worked. Thats weird. Thanks anyway

[kdpenner]

This is not just an issue while debugging. I have the inline backend activated. Here's a minimal working example:

import numpy as np
import matplotlib.pyplot as plt

x, y = np.random.rand(2, 5)

fig = plt.figure()

ax = fig.add_subplot(111)

ax.scatter(x, y)

If I "Run file" on the example, an inline plot appears. If I select the entirety of the example and "Run selection or current line", an inline plot appears. If I select lines 1-6 and "Run selection or current line", and type the rest of the example in the console, no plot appears. The reference to the plot object does appear.

[impact27]

This is not just an issue while debugging. I have the inline backend activated. Here's a minimal working example:

import numpy as np
import matplotlib.pyplot as plt

x, y = np.random.rand(2, 5)

fig = plt.figure()

ax = fig.add_subplot(111)

ax.scatter(x, y)

If I "Run file" on the example, an inline plot appears. If I select the entirety of the example and "Run selection or current line", an inline plot appears. If I select lines 1-6 and "Run selection or current line", and type the rest of the example in the console, no plot appears. The reference to the plot object does appear.

I don't think this is related to this bug. This is one limitation of using the inline back-end that the plot doesn't "redraw" itself. If you need to explicitly plot something in the inline back-end, you can do:

from IPython.core.display import display
display(fig)

[kdpenner]

display(fig) works. Still, the inconsistent behavior between fully running a file and partially running a file and finishing in the console is confusing.

[mmagnuski]

@kdpenner - you'll get the same behavior for example in notebook if you split the code between cells, this is expected. But I don't think you need to use display() function for this. Typing fig in the console should draw the figure.

[kdpenner]

fig works too. Maybe this difference in behaviors was designed, but it's not what I, as a user, expect.

[mmagnuski]

@kdpenner What I meant was that you'll get the same behavior in other environments with inline backend so this is not an issue with Spyder.

[kdpenner]

@mmagnuski Understood. Thanks for the fig workaround.

[mmagnuski]

🎉

[mhdadk]

Thanks a lot for fixing this!

[hmaarrfk]

exciting!

[ccordoba12]

You can read about this and the other improvements to our debugger that landed in version 4.2.0 here.