Open ouboub opened 2 years ago
If the matlab kernel is available on your machine via the jupyter kernelspec list
shell command and the ob-jupyter
library was loaded after that kernel was installed. You should just have to setup your source blocks like so
#+BEGIN_SRC jupyter-matlab :session m :async yes
x = [1, 2, 3, 4, 5];
fprintf('|%d', x)
#+END_SRC
The :kernel
header argument should already default to a valid kernel so no need to set it unless you have multiple matlab kernels.
If you installed the matlab kernel after loading the ob-jupyter
library, you need to give a call to (org-babel-jupyter-aliases-from-kernelspecs t)
so that those jupyter-matlab source blocks become available.
To change the default header args. you would set the org-babel-default-header-args:jupyter-matlab
variable.
Let me know if you run into any issues or if the support for the matlab kernel is lacking in some respects. I don't think I've come across anyone who uses the matlab kernel with this package.
"NN" == Nathaniel Nicandro @.***> writes:
If the matlab kernel is available on your machine via the
jupyter kernelspec list
shell command and theob-jupyter
library was loaded
jupyter kernelspec list returns
Available kernels: octave /home/oub/.local/share/jupyter/kernels/octave matlab /usr/local/share/jupyter/kernels/matlab python2 /usr/local/share/jupyter/kernels/python2 python3 /usr/local/share/jupyter/kernels/python3
after that kernel was installed.
I am not sure what you mean by the kernel was installed! Or do you mean the kernel was started via /usr/bib/python3
import matlab.engine eng = matlab.engine.start_matlab()
You should just have to setup your source blocks like so
#+BEGIN_SRC jupyter-matlab :session m :async yes x = [1, 2, 3, 4, 5]; fprintf('|%d', x) #+END_SRC
That did not work, I attach the error.
The
:kernel
header argument should already default to a valid kernel so no need to set it unless you have multiple matlab kernels.If you installed the matlab kernel after loading the
ob-jupyter
library, you need to give a call to(org-babel-jupyter-aliases-from-kernelspecs t)
so that those jupyter-matlab source blocks become available.
I executed that as well and executed the block, but got the same error message
To change the default header args. you would set the
org-babel-default-header-args:jupyter-matlab
variable.
I am not sure what you propose exactly
Let me know if you run into any issues or if the support for the matlab kernel is lacking in some respects. I don't think I've come across anyone who uses the matlab kernel with this package.
Here is the attached error, please tell me whether github let is pass.
Debugger entered--Lisp error: (error "Kernel did not respond to kernel-info request")
signal(error ("Kernel did not respond to kernel-info request"))
error("Kernel did not respond to kernel-info request")
jupyter-kernel-info(#
apply(#f(compiled-function (session kernel) #<bytecode -0x43d82c99df8b941>) (#s(org-babel-jupyter-session :name "m") "matlab"))
apply(#f(compiled-function (cl--cnm session kernel) "Rename the returned client's REPL buffer to include SESSION's name.\nAlso set `jupyter-include-other-output' to nil for the session so\nthat output produced by other clients do not get handled by the\nclient." #<bytecode 0xa99b0c1f14575a1>) #f(compiled-function (&rest cnm-args) #<bytecode 0xd41e4c82814101d>) (#s(org-babel-jupyter-session :name "m") "matlab"))
apply(#f(compiled-function (&rest args) #<bytecode -0x12e82d30adf230bd>) #s(org-babel-jupyter-session :name "m") "matlab") org-babel-jupyter-initiate-client(#s(org-babel-jupyter-session :name "m") "matlab") org-babel-jupyter-initiate-session-by-key("m" ((:colname-names) (:rowname-names) (:result-params "replace") (:result-type . value) (:results . "replace") (:exports . "code") (:cache . "no") (:noweb . "no") (:hlines . "no") (:tangle . "no") (:kernel . "matlab") (:session . "m") (:async . "yes"))) org-babel-jupyter-initiate-session("m" ((:colname-names) (:rowname-names) (:result-params "replace") (:result-type . value) (:results . "replace") (:exports . "code") (:cache . "no") (:noweb . "no") (:hlines . "no") (:tangle . "no") (:kernel . "matlab") (:session . "m") (:async . "yes"))) org-babel-execute:jupyter-matlab("x = [1, 2, 3, 4, 5];\nfprintf('|%d', x)" ((:colname-names) (:rowname-names) (:result-params "replace") (:result-type . value) (:results . "replace") (:exports . "code") (:cache . "no") (:noweb . "no") (:hlines . "no") (:tangle . "no") (:kernel . "matlab") (:session . "m") (:async . "yes"))) org-babel-execute-src-block(nil ("jupyter-matlab" "x = [1, 2, 3, 4, 5];\nfprintf('|%d', x)" ((:colname-names) (:rowname-names) (:result-params "replace") (:result-type . value) (:results . "replace") (:exports . "code") (:async . "yes") (:session . "m") (:kernel . "matlab") (:tangle . "no") (:hlines . "no") (:noweb . "no") (:cache . "no")) "" nil 1 "(ref:%s)")) org-ctrl-c-ctrl-c(nil) funcall-interactively(org-ctrl-c-ctrl-c nil) call-interactively(org-ctrl-c-ctrl-c nil nil) command-execute(org-ctrl-c-ctrl-c)
after that kernel was installed.
I am not sure what you mean by the kernel was installed!
I meant that the kernel was installed on your system and listed as an available kernel.
To change the default header args. you would set the
org-babel-default-header-args:jupyter-matlab
variable.I am not sure what you propose exactly
Not proposing anything, just giving the name of the variable to change the default header args.
Here is the attached error, please tell me whether github let is pass.
Yes github did let pass the backtrace.
It seems that the matlab kernel did not respond in time to the
kernel_info_request
message that was sent to it. That error happens
when it takes longer than 3x jupyter-long-timeout
seconds for the
kernel to respond. I doubt it's that emacs-jupyter
is not waiting long
enough for the kernel to startup since jupyter-long-timeout
is already
10 s, so I'll assume that there is something else going on.
Could you evaluate (setq jupyter--debug t)
, run the code block again,
and copy here the debug messages printed in the *Messages*
buffer.
"NN" == Nathaniel Nicandro @.***> writes:
after that kernel was installed.
I am not sure what you mean by the kernel was installed!
I meant that the kernel was installed on your system and listed as an available kernel.
It was.
To change the default header args. you would set the
org-babel-default-header-args:jupyter-matlab
variable.I am not sure what you propose exactly
Not proposing anything, just giving the name of the variable to change the default header args.
Here is the attached error, please tell me whether github let is pass.
Yes github did let pass the backtrace.
It seems that the matlab kernel did not respond in time to the
kernel_info_request
message that was sent to it. That error happens when it takes longer than 3xjupyter-long-timeout
seconds for the kernel to respond. I doubt it's thatemacs-jupyter
is not waiting long enough for the kernel to startup sincejupyter-long-timeout
is already 10 s, so I'll assume that there is something else going on.
Just one piece of additional information that might be relevant, when I use the kernel via the ob-ipython package, and execute the src block (with the ipython setting), it first needs almost 30 sec to come up, then however it is quite fast. Maybe that is a particular feature in my Ubuntu distribution/python version.
Could you evaluate
(setq jupyter--debug t)
, run the code block again, and copy here the debug messages printed in the*Messages*
buffer.
Here is what I executed:
(setq jupyter--debug t)
(org-babel-jupyter-aliases-from-kernelspecs t)
#+BEGIN_SRC jupyter-matlab :session m :async yes
x = [1, 2, 3, 4, 5];
fprintf('|%d', x)
#+END_SRC
Here it comes, I also attach the bug again, but it seems the same
executing Jupyter-Matlab code block...
jupyter-start-kernel: default-directory = /home/oub/Mail/Remember/
jupyter-start-kernel: Starting process with args "python -m matlab_kernel -f /home/oub/.local/share/jupyter/runtime/emacs-kernel-xsfD0T.json"
Starting matlab kernel process...done
SENDING: :kernel-info-request ceeb4686-d417-43ca-886d-c67f1af5c7c8 nil
SENT: (:shell ceeb4686-d417-43ca-886d-c67f1af5c7c8)
Requesting kernel info...done
Entering debugger...
Auto-saving...done
Mark set
Copied text until "l)
command-execute(org-ctrl-c-ctrl-c)
"
(New file)
Mark set
Saving file /home/oub/Mail/Remember/jup-bug2.txt...
Wrote /home/oub/Mail/Remember/jup-bug2.txt
Back to top level
finalizer failed: (cl-no-primary-method jupyter-kill-kernel #s(jupyter-kernel-process-manager (#
Debugger entered--Lisp error: (error "Kernel did not respond to kernel-info request")
signal(error ("Kernel did not respond to kernel-info request"))
error("Kernel did not respond to kernel-info request")
jupyter-kernel-info(#
apply(#f(compiled-function (session kernel) #<bytecode -0x43d82c99df8b941>) (#s(org-babel-jupyter-session :name "m") "matlab"))
apply(#f(compiled-function (cl--cnm session kernel) "Rename the returned client's REPL buffer to include SESSION's name.\nAlso set `jupyter-include-other-output' to nil for the session so\nthat output produced by other clients do not get handled by the\nclient." #<bytecode 0xa99b0c1f14575a1>) #f(compiled-function (&rest cnm-args) #<bytecode 0xd41e4c82814101d>) (#s(org-babel-jupyter-session :name "m") "matlab"))
apply(#f(compiled-function (&rest args) #<bytecode -0x12e82d30adf230bd>) #s(org-babel-jupyter-session :name "m") "matlab") org-babel-jupyter-initiate-client(#s(org-babel-jupyter-session :name "m") "matlab") org-babel-jupyter-initiate-session-by-key("m" ((:colname-names) (:rowname-names) (:result-params "replace") (:result-type . value) (:results . "replace") (:exports . "code") (:cache . "no") (:noweb . "no") (:hlines . "no") (:tangle . "no") (:kernel . "matlab") (:session . "m") (:async . "yes"))) org-babel-jupyter-initiate-session("m" ((:colname-names) (:rowname-names) (:result-params "replace") (:result-type . value) (:results . "replace") (:exports . "code") (:cache . "no") (:noweb . "no") (:hlines . "no") (:tangle . "no") (:kernel . "matlab") (:session . "m") (:async . "yes"))) org-babel-execute:jupyter-matlab("x = [1, 2, 3, 4, 5];\nfprintf('|%d', x)" ((:colname-names) (:rowname-names) (:result-params "replace") (:result-type . value) (:results . "replace") (:exports . "code") (:cache . "no") (:noweb . "no") (:hlines . "no") (:tangle . "no") (:kernel . "matlab") (:session . "m") (:async . "yes"))) org-babel-execute-src-block(nil ("jupyter-matlab" "x = [1, 2, 3, 4, 5];\nfprintf('|%d', x)" ((:colname-names) (:rowname-names) (:result-params "replace") (:result-type . value) (:results . "replace") (:exports . "code") (:async . "yes") (:session . "m") (:kernel . "matlab") (:tangle . "no") (:hlines . "no") (:noweb . "no") (:cache . "no")) "" nil 73 "(ref:%s)")) org-ctrl-c-ctrl-c(nil) funcall-interactively(org-ctrl-c-ctrl-c nil) call-interactively(org-ctrl-c-ctrl-c nil nil) command-execute(org-ctrl-c-ctrl-c)
It seems that the matlab kernel did not respond in time to the
kernel_info_request
message that was sent to it. That error happens when it takes longer than 3xjupyter-long-timeout
seconds for the kernel to respond. I doubt it's thatemacs-jupyter
is not waiting long enough for the kernel to startup sincejupyter-long-timeout
is already 10 s, so I'll assume that there is something else going on.Just one piece of additional information that might be relevant, when I use the kernel via the ob-ipython package, and execute the src block (with the ipython setting), it first needs almost 30 sec to come up, then however it is quite fast. Maybe that is a particular feature in my Ubuntu distribution/python version.
Or maybe it has to do with MATLAB itself taking a long time to initialize? I do remember when I was back at university MATLAB was one of those applications that took a long time to start, at least on the Windows machines that I had to run it on.
It makes sense that that initialization time would apply also to the Jupyter kernel since it just starts up a MATLAB process in the background (based on the documentation at https://www.mathworks.com/help/matlab/apiref/matlab.engine.start_matlab.html).
Here it comes, I also attach the bug again, but it seems the same
The debug messages do show at least that the kernel_info_request
is
being sent, as far as Emacs can tell, and that the kernel is not sending
any messages to Emacs within the 3x jupyter-long-timeout
period that
emacs-jupyter
waits to receive a kernel_info_reply
.
So I'm now more suspicious that this does indeed have to do with not
waiting long enough for the kernel to initialize. Have you had any luck
increasing jupyter-long-timeout
to 15 or higher?
Also it seems that there are two different MATLAB Jupyter kernels that
are popular: imatlab (https://github.com/imatlab/imatlab) and
matlab_kernel (https://github.com/calysto/matlab_kernel). My suspicion
depends on which kernel you use. I browsed the code of both
repos. imatlab starts a MATLAB process during the __init__
method of
the kernel class and so that initialization would have to finish before
the kernel could respond to a kernel_info_request
(and increasing
jupyter-long-timeout
would hopefully get you past the error if you
used this kernel). The matlab_kernel starts a MATLAB process on demand
when evaluating code or responding to messages other than a
kernel_info_request
and so should be able to respond to a
kernel_info_request
right away. Which of those two are you using or
are you using some other MATLAB kernel?
"NN" == Nathaniel Nicandro @.***> writes:
It seems that the matlab kernel did not respond in time to the
kernel_info_request
message that was sent to it. That error happens when it takes longer than 3xjupyter-long-timeout
seconds for the kernel to respond. I doubt it's thatemacs-jupyter
is not waiting long enough for the kernel to startup sincejupyter-long-timeout
is already 10 s, so I'll assume that there is something else going on.Just one piece of additional information that might be relevant, when I use the kernel via the ob-ipython package, and execute the src block (with the ipython setting), it first needs almost 30 sec to come up, then however it is quite fast. Maybe that is a particular feature in my Ubuntu distribution/python version.
Or maybe it has to do with MATLAB itself taking a long time to initialize? I do remember when I was back at university MATLAB was one of those applications that took a long time to start, at least on the Windows machines that I had to run it on.
It makes sense that that initialization time would apply also to the Jupyter kernel since it just starts up a MATLAB process in the background (based on the documentation at https://www.mathworks.com/help/matlab/apiref/matlab.engine.start_matlab.html).
Here it comes, I also attach the bug again, but it seems the same
The debug messages do show at least that the
kernel_info_request
is being sent, as far as Emacs can tell, and that the kernel is not sending any messages to Emacs within the 3xjupyter-long-timeout
period thatemacs-jupyter
waits to receive akernel_info_reply
.So I'm now more suspicious that this does indeed have to do with not waiting long enough for the kernel to initialize. Have you had any luck increasing
jupyter-long-timeout
to 15 or higher?
No, I will do this now.
I set it to 60
But it did not help.
Also it seems that there are two different MATLAB Jupyter kernels that
You mean besides the one shipped with matworks?
are popular: imatlab (https://github.com/imatlab/imatlab) and matlab_kernel (https://github.com/calysto/matlab_kernel).
My suspicion depends on which kernel you use. I browsed the code of both repos. imatlab starts a MATLAB process during the
__init__
method of the kernel class and so that initialization would have to finish before the kernel could respond to akernel_info_request
(and increasingjupyter-long-timeout
would hopefully get you past the error if you used this kernel). The matlab_kernel starts a MATLAB process on demand when evaluating code or responding to messages other than akernel_info_request
and so should be able to respond to akernel_info_request
right away. Which of those two are you using or are you using some other MATLAB kernel?
The second one I use, the one provided my calysto (as I said it works with ob-ipython (but I wanted to see whether your package works better for example supports the plot command)
regards
Also it seems that there are two different MATLAB Jupyter kernels that
You mean besides the one shipped with matworks?
Yes if they ship a Jupyter kernel. I did not find one when I looked. I didn't look very hard.
The second one I use, the one provided my calysto (as I said it works with ob-ipython
OK, I'll have to figure out what is going on then. Usually the debug messages give some hint, but it doesn't look like the kernel is sending back anything in this case.
Just to be sure, can you run a jupyter-python source block without problem? If you run the source block:
#+begin_src jupyter-python :session py
print("hello")
#+end_src
Do you get the result:
#+begin_src jupyter-python :session py
print("hello")
#+end_src
#+RESULTS:
: hello
Since I don't have access to a MATLAB license, I am unable to reproduce
the error with matlab_kernel
. However, I did install octave_kernel
,
by the same author, just to rule out any bad interactions between
emacs-jupyter
and the kernel Python class the author uses in both
those projects. I was able to successfully execute source blocks that
used the Octave kernel, so I don't think the issue has anything to do
with that Python class.
If possible, can you try the imatlab kernel to see if that works? Just to isolate the issue further to something to do with matlab_kernel.
It's strange that emacs-jupyter
doesn't work, but ob-ipython
does. After I refresh my memory on what ob-ipython
is doing to talk to
kernels I'll open up an issue with matlab_kernel
to see if we can make
progress on finding out what is going on.
"NN" == Nathaniel Nicandro @.***> writes:
Also it seems that there are two different MATLAB Jupyter kernels that
You mean besides the one shipped with matworks?
Yes if they ship a Jupyter kernel. I did not find one when I looked. I didn't look very hard.
The second one I use, the one provided my calysto (as I said it works with ob-ipython
OK, I'll have to figure out what is going on then. Usually the debug messages give some hint, but it doesn't look like the kernel is sending back anything in this case.
Just to be sure, can you run a jupyter-python source block without problem? If you run the source block:
#+begin_src jupyter-python :session py print("hello") #+end_src
Do you get the result:
#+begin_src jupyter-python :session py print("hello") #+end_src #+RESULTS: : hello
YES, that works
Since I don't have access to a MATLAB license, I am unable to reproduce the error with
matlab_kernel
. However, I did installoctave_kernel
, by the same author, just to rule out any bad interactions betweenemacs-jupyter
and the kernel Python class the author uses in both those projects. I was able to successfully execute source blocks that used the Octave kernel, so I don't think the issue has anything to do with that Python class.If possible, can you try the imatlab kernel to see if that works? Just to isolate the issue further to something to do with matlab_kernel.
Hm, I hesitate a bit to be honest. The reason is that it took me always a week to make ob-ipython to work, because my python 3.5 was a bit screwed up.
In that context I have question to you (since I don't use octave)
If your run the octave equivalent of
clear all close all t=[0:0.1:1]; y=sin(t); figure(1) plot(t,y) print -dpng simple.png
Does the plot figure pop up? In my case with ob-ipython it does not, this is why I want to give emacs-jupyter a try. If you now tell me it does not work neither, then I would prefer for the moment not to touch my working matlab_kernel installation.
If it does, I will try it out.
It's strange that
emacs-jupyter
doesn't work, butob-ipython
does. After I refresh my memory on whatob-ipython
is doing to talk to kernels I'll open up an issue withmatlab_kernel
to see if we can make progress on finding out what is going on.
If your run the octave equivalent of
+begin_src matlab :results output latex :exports code :eval never-export :wrap latex
clear all close all t=[0:0.1:1]; y=sin(t); figure(1) plot(t,y) print -dpng simple.png
+end_src
Does the plot figure pop up?
If I run
#+begin_src jupyter-octave :session octave-test :results output latex :exports code :eval never-export :wrap latex
%plot -b inline:gnuplot
clear all
close all
t=[0:0.1:1];
y=sin(t);
figure(1)
plot(t,y)
print -dpng simple.png
#+end_src
(NOTE: I had to also install gnuplot and use the plot magic "%plot -b inline:gnuplot", see https://github.com/Calysto/octave_kernel#configuration)
I get as the result
#+RESULTS:
#+begin_latex
[[file:./.ob-jupyter/ea43ecbeafc0603e0e9a18d2ce51261f97b3d4ac.png]]
#+end_latex
And, since I have something like org-display-inline-images
as a hook
in org-babel-after-execute-hook
I get the image to display after
executing.
Not sure if an Org file link is allowed to be embedded in a LaTeX export
block, but when I did a LaTeX export to PDF, changing the :exports code
header argument to :exports both
, the image did appear in the
exported file.
Instead of that last line "print -dpng simple.png" you can remove it and
specify the header argument :file simple.png
and then the result would
be
#+RESULTS:
#+begin_latex
[[file:simple.png]]
#+end_latex
If you do plan on trying out emacs-jupyter
with the Octave kernel, let
me know of any issues. I did find that textual output produced carriage
return characters, ^M, at the end of lines.
#+begin_src jupyter-octave :session octave-test
rand(4, 4)
#+end_src
#+RESULTS:
: ans =^M
: ^M
: 7.4726e-01 5.4297e-01 2.3448e-01 9.4782e-03^M
: 5.0026e-01 6.2880e-01 9.4922e-02 5.3403e-01^M
: 2.7600e-02 2.3409e-01 3.8434e-01 2.6925e-01^M
: 2.6792e-02 2.9401e-01 2.0709e-01 6.8352e-01^M
: ^M
Please be aware too that supporting all the various combinations of Org
header arguments is a work in progress. For example, I'm not sure
:wrap
will do what is expected in combination with :async yes
(evaluate the source block asynchronously).
-- Nathaniel
"NN" == Nathaniel Nicandro @.***> writes:
If your run the octave equivalent of
+begin_src matlab :results output latex :exports code :eval never-export :wrap latex
clear all close all t=[0:0.1:1]; y=sin(t); figure(1) plot(t,y) print -dpng simple.png
+end_src
Does the plot figure pop up?
If I run
#+begin_src jupyter-octave :session octave-test :results output latex :exports code :eval never-export :wrap latex %plot -b inline:gnuplot clear all close all t=[0:0.1:1]; y=sin(t); figure(1) plot(t,y) print -dpng simple.png #+end_src
(NOTE: I had to also install gnuplot and use the plot magic "%plot -b inline:gnuplot", see https://github.com/Calysto/octave_kernel#configuration)
I get as the result
#+RESULTS: #+begin_latex [[file:./.ob-jupyter/ea43ecbeafc0603e0e9a18d2ce51261f97b3d4ac.png]] #+end_latex
I think I did not express myself clearly enough. What I mean is does a figure pop up showing the plot. I am asking because generating a png is time consuming and the result might be wrong this is why I would like to see the figure itself. When I call this command above from the emacs-matlab shell, the figure pops up, but not when I use a scr block and ob-ipython. This is why I am asking whether your package makes the figure visible, with octave.
I attach a screenshot of what I mean maybe this clarify things
What I mean is does a figure pop up showing the plot. I am asking because generating a png is time consuming and the result might be wrong this is why I would like to see the figure itself. When I call this command above from the emacs-matlab shell, the figure pops up, but not when I use a scr block and ob-ipython. This is why I am asking whether your package makes the figure visible, with octave.
I attach a screenshot of what I mean maybe this clarify things
The attached screenshot did not come through.
But I think what you are asking is if a figure window comes up showing the plot. Yes, a figure window does come up with the code that I showed, except I had to replace the "%plot -b inline:gnuplot" line with "%plot -b fltk".
I had initially thought you were asking if the plot image could be displayed in the Org buffer.
-- Nathaniel
"NN" == Nathaniel Nicandro @.***> writes:
What I mean is does a figure pop up showing the plot. I am asking because generating a png is time consuming and the result might be wrong this is why I would like to see the figure itself. When I call this command above from the emacs-matlab shell, the figure pops up, but not when I use a scr block and ob-ipython. This is why I am asking whether your package makes the figure visible, with octave.
I attach a screenshot of what I mean maybe this clarify things
The attached screenshot did not come through.
But I think what you are asking is if a figure window comes up showing the plot. Yes, a figure window does come up with the code that I showed, except I had to replace the "%plot -b inline:gnuplot" line with "%plot -b fltk".
Aha this is precisly what I am looking for. I will investigate this further and if I am still unable to get this in my current setting I will try the other kernel and come back to you.
By the way what is -b fltk supposed to be?
By the way what is -b fltk supposed to be?
The -b stands for the plotting backend (see https://github.com/Calysto/octave_kernel#configuration) and fltk is the name of the backend which was default on my installation of Octave and which I got to show a figure window. You can see the available backends through the "available_graphics_toolkits()" command.
-- Nathaniel
Hi
so far I have used ob-ipython (Ubuntu 16.04, matlab 2019a python 3.5) and the following setting worked
and then
What is the equivalent for jupyter?
I tried
What do I miss?