Closed dhermes closed 7 years ago
That was my laptop. Works fine on my desktop (previous stacktrace was erroneous).
It seems causing the entire laptop to hang is related to windowing / UI frameworks but I'm not very familiar with them.
As a (potentially helpful) start, I have the following matplotlib
plotting backends:
OPTIONAL BACKEND EXTENSIONS
macosx: no [Mac OS-X only]
qt5agg: no [PyQt5 not found]
qt4agg: yes [installing, Qt: 4.8.6, PyQt: 4.8.6]
pyside: yes [installing, Qt: 4.8.6, PySide: 1.2.1]
gtk3agg: yes [installing, version 3.8.10]
gtk3cairo: yes [installing, version 3.8.10]
gtkagg: yes [installing, Gtk: 2.24.23 pygtk: 2.24.0]
tkagg: yes [installing, version 81008]
wxagg: yes [installing, version 2.8.12.1]
gtk: yes [installing, Gtk: 2.24.23 pygtk: 2.24.0]
agg: yes [installing]
cairo: yes [installing, pycairo version 1.8.8]
windowing: no [Microsoft Windows only]
Sorry @dhermes, I just saw this Issue report. Do you have gnuplot-x11
installed?
Yes:
$ dpkg -s gnuplot-x11
Package: gnuplot-x11
Status: install ok installed
Priority: optional
Section: math
...
Should it be un-installed? (i.e. is that a good thing or a bad thing)
Hmm, can you try firing up just ipython
(no notebook) and running %paste
with this code on your clipboard:
from oct2py import octave
octave.eval("""
subplot(121);
[x, y] = meshgrid(0:0.1:3);
r = sin(x - 0.5).^2 + cos(y - 0.5).^2;
surf(x, y, r);
subplot(122);
sombrero()
""")
@blink1073 It works just fine from IPython.
How about this (from ipython), copy this to clipboard:
%%octave -g
subplot(121);
[x, y] = meshgrid(0:0.1:3);
r = sin(x - 0.5).^2 + cos(y - 0.5).^2;
surf(x, y, r);
subplot(122);
sombrero()
And run this IPython session:
In [1]: %load_ext oct2py.ipython
In [2]: %paste
No issues there either.
I need to re-run the offending notebook (haven't since 11/6). It may be a heisenbug based on some transient kernel / install issue that no longer exists.
I will update after trying (it may take awhile if I have to reboot my machine).
UPDATE: @blink1073 it still brings my machine to its knees.
Sometimes I have to kill my .ipython
folder to clear up notebook issues.
On Fri, Dec 19, 2014 at 8:45 PM, Danny Hermes notifications@github.com wrote:
No issues there either.
I need to re-run the offending notebook (haven't since 11/6). It may be a heisenbug based on some transient kernel / install issue that no longer exists.
I will update after trying (it may take awhile if I have to reboot my machine).
— Reply to this email directly or view it on GitHub https://github.com/blink1073/oct2py/issues/61#issuecomment-67722291.
I blew up ~/.ipython
and it still brought the machine to its knees.
The %%octave -s 500,500
statements are what cause the issue. I isolated this for both of the cells (butterworth filter and sombrero) and got the same behavior.
Does using the size argument have the same effect in the console (just adding it to my example above)?
@blink1073 The size argument is the deal breaker. It caused the same issue from the console. I still had to reboot.
I tried an almost identical setup on my machine with no failures:
$ /usr/bin/ipython --version
2.3.1
silvester: ~
$ sudo pip show oct2py
---
Name: oct2py
Version: 2.4.0
Location: /usr/local/lib/python2.7/dist-packages
Requires:
silvester: ~
$ cat /etc/lsb-release
DISTRIB_ID=LinuxMint
DISTRIB_RELEASE=17
DISTRIB_CODENAME=qiana
DISTRIB_DESCRIPTION="Linux Mint 17 Qiana"
silvester: ~
$ octave --version
GNU Octave, version 3.8.1
Perhaps try uncommenting this line (it should be commented in the released version) and run again from the console?
Yes I have a Desktop set up identically with no issues. It has to be some low-level library that either isn't installed on one machine or that doesn't work with the drivers on the laptop.
I adding the logging and tried a few things.
I ran just from oct2py import octave
and the only thing logged was
'NoneType' object has no attribute 'terminate'
on exit. (This was the same if I executed octave.eval(...)
after the import.)
Just loading %load_ext oct2py.ipython
resulted in
'NoneType' object has no attribute 'write'
'NoneType' object has no attribute 'terminate'
on exit. Executing the sombrero plot with -g
(which works fine) I got a large log output but it doesn't seem to be insightful.
Executing with -s
(the butterworth snippet from the example notebook) the log was identical except for the code snippet. (I didn't get a chance to capture because the computer froze.)
Yep, I'm at a loss. The only thing I can think of that might do it is setting the DefaultFigurePosition might not work with your setup. You could try commenting out those lines and see if it works.
I'm happy to dig into the source to debug, I'm just unfamiliar with how -g
and -s
pass from IPython to your extension. Can you provide some pointers?
When you pass -s
, the magic function passes the height and width to Oct2Py.eval
, which then gets the appropriate plotting commands from Oct2Py._get_plot_commands
. This function adds Octave commands to set the figure size (using DefaultFigurePostion), and to optionally save the figure to a file (if the plot dir was set).
If the -g
flag is not set, the plot_dir
is set to a temp directory, which the magic function reads and passes to IPython, and the Octave plot window is not shown. I don't think the -g
flag is your problem, but you could try leaving it out.
So two interesting things have happened.
_Session.evaluate
, particularly self.write
when the output is the command wrapped with some other stuff.Any thoughts?
I have no idea about your second item, but there's a couple more things we can check. How about:
from oct2py import octave
octave.eval("set(0, 'DefaultFigurePosition', [300, 200, 500, 500]);")
print('one')
octave.eval("__oct2py_figures = [];")
print('two')
octave.eval("close all;")
print('three')
octave.eval("__oct2py_figure_visible = 'off';")
print('four')
octave.eval("plot([1,2,3])")
print('five')
(preferrably from the IPython prompt using %paste
)
That works just fine and creates a plot. (With Chrome open, whatever issue that might be responsible for.)
Grasping at straws (trying to mimic the OctaveMagic
function call)...
from oct2py import octave
octave.eval("set(0, 'DefaultFigurePosition', [300, 200, 500, 500]);", return_both=True, verbose=False)
print('one')
octave.eval("__oct2py_figures = [];", return_both=True, verbose=False)
print('two')
octave.eval("close all;", return_both=True, verbose=False)
print('three')
octave.eval("__oct2py_figure_visible = 'off';", return_both=True, verbose=False)
print('four')
octave.eval("plot([1,2,3])", return_both=True, verbose=False)
print('five')
ISTM the issue is process contention and I don't fully understand what is happening when you write to an active process:
def write(self, message):
"""Write a message to the process using utf-8 encoding"""
os.write(self.wfid, message.encode('utf-8'))
Also, I captured the output after using -g
and it seems that
__oct2py_figures
never gets populated in the -g
case or the -s
case-s
doesn't make a plot from the console because there is no drawnow("expose")
statement (this is probably not true)write
method is the same one used by pexpect
when writing to a child process stdin. I don't think that's the problem, but IPC is a black art.__oct2py_figures
is getting populated by not using -g
and adding disp(outfile)
after the outfile = sprintf(...)
line in core.py
. drawnow
is tied to -g
, not -s
(it should be there when -g
is set, because it forces the Octave plot window to draw).Hi @dhermes, would you care to try this with the latest version (3.5) and see if it works for you?
Sorry @dhermes didn't reply. I'm in a similar scenario to him and still seeing a hang. I saw your instructions above, and got it unstuck using -g. However, now it matches the failure that I get in my linux installation.
Thoughts? (Please be a bit patient- I'm likely to only be able to test a solution this weekend).
@josephcslater, I'll look at this after we get https://github.com/Calysto/octave_kernel/issues/50 fixed.
Thanks. Let me know if I can test anything.
@josephcslater, your issue should be fixed by upgrading oct2py to v3.5.10.
Unfortunately, it's still hanging.
%load_ext oct2py.ipython
%%octave
a = 1:10
disp(a)
Just hangs.
Likewise
x = %octave [1 2; 3 4];
x
hangs.
How do I verify that the update "stuck" and I am loading the new extension?
I just tried it again on Ubuntu 14.04, and it is working fine, can you verify the following?
In [1]: import oct2py; oct2py.__version__
Out[1]: '3.5.10'
Confirmed.
Works fine on my Ubuntu installation as well. Failing on my Mac.
Ah, I had to follow the "Using Aquaterm with Gnuplot" from http://wiki.octave.org/Octave_for_MacOS_X to get it to work on my Mac.
I'm not even trying to plot. Should this matter?
Yes because oct2py doesn't know if you are trying to plot until after you run the command, so we treat every command as a possible plotting command.
Hmmm, so this may not work for me. What I have working is octave 4.0.3 installed using macports (not brew). It plots via xterm, not aquaterm . Not sure where to go from here.
I don't think this is my problem. I just ran gnu plot, changed the terminal to aqua (set term aqua
) and plotting followed fine. So, gnu plot will recognize aqua. It may be that I need to set the default to aqua for oct2py interacting with it to work?
That shouldn't be required, AFAIK. Are you able to plot from octave-cli
without changing any settings?
Yes.
At least this isn't (so far?) the face plant of my issue #50 (I owe you a beer for that level of stupid on my part!).
I'm not entirely sure where to go from here. I am not keen to install Macports and spending another round of debugging the inner loop at the moment.
I get it. Have an idea how I could trace it?
It's not a strong need of mine right now, but more of a "this would be nice to have", so I appreciate how much you've already put into it.
You could try putting print statements in the inner evaluate
function, that is essentially what I ended up doing this morning to debug the Homebrew plotting. I'll be afk for the rest of the day.
Thank you, and enjoy the time away.
Sent from my iPhone
On Nov 19, 2016, at 12:14 PM, Steven Silvester notifications@github.com wrote:
You could try putting print statements in the inner evaluate function, that is essentially what I ended up doing this morning to debug the Homebrew plotting. I'll be afk for the rest of the day.
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.
Okay, I think the problem is trying to do control flow logic (for loops in this case) through the pipe, which is too brittle. v3.6.0 adds a separate _make_figs.m
function that encapsulates the logic. Do you mind testing that release?
Works! Thanks!
That you would ask if I mind... I owe it to you (and other freeware authors (which I am as well) to do so. Thank you for coming back to this.
FWIW: The failure happened even when no figures were involved. Further, figures are indeed working.
In light of this: I thought I should test your demo ipynb, it unfortunately failed on the cell
%%octave -s 500,500
# butterworth filter, order 2, cutoff pi/2 radians
b = [0.292893218813452 0.585786437626905 0.292893218813452];
a = [1 0 0.171572875253810];
freqz(b, a, 32);
the fail results being:
---------------------------------------------------------------------------
ExpatError Traceback (most recent call last)
<ipython-input-10-bca0f2444242> in <module>()
----> 1 get_ipython().run_cell_magic('octave', '-s 500,500', '\n# butterworth filter, order 2, cutoff pi/2 radians\nb = [0.292893218813452 0.585786437626905 0.292893218813452];\na = [1 0 0.171572875253810];\nfreqz(b, a, 32);')
/opt/local/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/IPython/core/interactiveshell.py in run_cell_magic(self, magic_name, line, cell)
2113 magic_arg_s = self.var_expand(line, stack_depth)
2114 with self.builtin_trap:
-> 2115 result = fn(magic_arg_s, cell)
2116 return result
2117
<decorator-gen-124> in octave(self, line, cell, local_ns)
/opt/local/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/IPython/core/magic.py in <lambda>(f, *a, **k)
186 # but it's overkill for just that one bit of state.
187 def magic_deco(arg):
--> 188 call = lambda f, *a, **k: f(*a, **k)
189
190 if callable(arg):
/Users/jslater/Library/Python/3.5/lib/python/site-packages/oct2py/ipython/octavemagic.py in octave(self, line, cell, local_ns)
337 if plot_format == 'svg':
338 image = self._fix_gnuplot_svg_size(image, size=(plot_width,
--> 339 plot_height))
340 display_data.append((key, {plot_mime_type: image}))
341
/Users/jslater/Library/Python/3.5/lib/python/site-packages/oct2py/ipython/octavemagic.py in _fix_gnuplot_svg_size(self, image, size)
109
110 """
--> 111 (svg,) = minidom.parseString(image).getElementsByTagName('svg')
112 viewbox = svg.getAttribute('viewBox').split(' ')
113
/opt/local/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/xml/dom/minidom.py in parseString(string, parser)
1966 if parser is None:
1967 from xml.dom import expatbuilder
-> 1968 return expatbuilder.parseString(string)
1969 else:
1970 from xml.dom import pulldom
/opt/local/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/xml/dom/expatbuilder.py in parseString(string, namespaces)
923 else:
924 builder = ExpatBuilder()
--> 925 return builder.parseString(string)
926
927
/opt/local/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/xml/dom/expatbuilder.py in parseString(self, string)
221 parser = self.getParser()
222 try:
--> 223 parser.Parse(string, True)
224 self._setup_subset(string)
225 except ParseEscape:
ExpatError: not well-formed (invalid token): line 189, column 92
Now hanging is:
%%octave -s 600,200 -f png
# Note: On Windows, this will not show the plots unless Ghostscript is installed.
subplot(121);
[x, y] = meshgrid(0:0.1:3);
r = sin(x - 0.5).^2 + cos(y - 0.5).^2;
surf(x, y, r);
subplot(122);
sombrero()
The final cell (using the octave plotting GUI) works fine.
Of course, order matters. It can't be executed with the others hanging.
Also hanging is:
%%octave -f svg
# Note: On Windows, this will not show the plots unless Ghostscript is installed.
subplot(121);
[x, y] = meshgrid(0:0.1:3);
r = sin(x - 0.5).^2 + cos(y - 0.5).^2;
surf(x, y, r);
subplot(122);
sombrero()
Interrupting yields the following. I have no idea of the value:
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-3-7e66b99b039b> in <module>()
----> 1 get_ipython().run_cell_magic('octave', '-f svg', '# Note: On Windows, this will not show the plots unless Ghostscript is installed.\n\nfigure(1)\nsubplot(121);\n[x, y] = meshgrid(0:0.1:3);\nr = sin(x - 0.5).^2 + cos(y - 0.5).^2;\nsurf(x, y, r);\n\nsubplot(122);\nsombrero()')
/opt/local/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/IPython/core/interactiveshell.py in run_cell_magic(self, magic_name, line, cell)
2113 magic_arg_s = self.var_expand(line, stack_depth)
2114 with self.builtin_trap:
-> 2115 result = fn(magic_arg_s, cell)
2116 return result
2117
<decorator-gen-124> in octave(self, line, cell, local_ns)
/opt/local/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/IPython/core/magic.py in <lambda>(f, *a, **k)
186 # but it's overkill for just that one bit of state.
187 def magic_deco(arg):
--> 188 call = lambda f, *a, **k: f(*a, **k)
189
190 if callable(arg):
/Users/jslater/Library/Python/3.5/lib/python/site-packages/oct2py/ipython/octavemagic.py in octave(self, line, cell, local_ns)
305 plot_name=plot_name,
306 verbose=False,
--> 307 return_both=True)
308 except oct2py.Oct2PyError as exception:
309 msg = str(exception)
ValueError: too many values to unpack (expected 2)
However, it works fine with the -g
option.
Thanks again for the reports @josephcslater, I'll try to sneak in some time to work on this over the holiday.
Running the provided sample causes my entire machine (not just the example) to hang when using the subplot examples.
Machine info: