Closed mbujosab closed 4 months ago
jkitchin
commented
5 months ago Good news for me, I was able to get it working again by running M-x jupyter-refresh-kernelspecs. The reason it was failing for me is there was an out of date kernelspec that was trying to load an older version of R that no longer existed.
mbujosab
commented
5 months ago Maybe it is because I have an old version of Scimax, but I can't find the function M-x jupyter-refresh-kernelspecs.
mbujosab
commented
5 months ago When I try to run an R code block, I get the following message:
ess-mode: Customise alist is not specified, nor ess-local-customize-alist is set
jkitchin
commented
5 months ago jupyter-refresh-kernelspecs is an autoloaded interactive byte-compiled Lisp function in ‘jupyter-kernelspec.el’.
I have jupyter-20231204.529 installed. It also seems to exist in the most recent package.
mbujosab
commented
5 months ago Dear Professor Kitchin,
I have installed a new version of Scimax (I did a manual installation by cloning the GitHub repository). During the process, I encountered several warning errors and another error when I tried to load 'ox-ipynb'. (Error (use-package): Cannot load ox-ipynb)
I am not sure what I did wrong.
Error (use-package): Failed to install aggressive-indent: Assertion failed: (or (= (buffer-size tar-data-buffer) (buffer-size)) (eq tar-data-swapped (> (buffer-size tar-data-buffer) (buffer-size)))) Error (use-package): Cannot load aggressive-indent Error (use-package): Failed to install citeproc: https://elpa.gnu.org/packages/queue-0.2.el: Not found Error (use-package): Cannot load citeproc Error (use-package): Failed to install org-ref: https://elpa.gnu.org/packages/queue-0.2.el: Not found Error (use-package): Cannot load org-ref Error (use-package): Failed to parse package org-ref-ivy: Wrong type argument: stringp, nil Error (use-package): Failed to install rainbow-mode: Failed to verify signature: "rainbow-mode-1.0.6.tar.sig" Error (use-package): Cannot load rainbow-mode Error (use-package): Failed to install undo-tree: https://elpa.gnu.org/packages/queue-0.2.el: Not found Error (use-package): Cannot load undo-tree Error (use-package): scimax-email/:catch: Cannot open load file: No existe el fichero o el directorio, org-ref-export Error (use-package): Failed to install org-ql: Failed to verify signature: "peg-1.0.1.tar.sig" Error (use-package): Cannot load org-ql
When I try to run the code in
work misspelled. Type C-; to fix it.
Executing Jupyter-R unknown at position 1313...
Launching ir kernel...
Starting ir kernel process...done
Launching ir kernel...done
Requesting kernel info...
Loading /home/marcos/.emacs.d/elpa/zmq-20230608.1856/emacs-zmq (module)...
zmq-load: Module could not be opened: "/home/marcos/.emacs.d/elpa/zmq-20230608.1856/emacs-zmq.so", "/lib/x86_64-linux-gnu/libc.so.6: version GLIBC_2.33' not found (required by /home/marcos/.emacs.d/elpa/zmq-20230608.1856/emacs-zmq.so)" Executing Emacs-Lisp unknown at position 694... Code block produced no output. Executing Jupyter-R unknown at position 1313... Launching ir kernel... Starting ir kernel process...done Launching ir kernel...done Requesting kernel info... Loading /home/marcos/.emacs.d/elpa/zmq-20230608.1856/emacs-zmq (module)... zmq-load: Module could not be opened: "/home/marcos/.emacs.d/elpa/zmq-20230608.1856/emacs-zmq.so", "/lib/x86_64-linux-gnu/libc.so.6: versionGLIBC_2.33' not found (required by /home/marcos/.emacs.d/elpa/zmq-20230608.1856/emacs-zmq.so)"
Mark set [5 times]
With the new version of scimax It seems that the R-kernel runs (but I have to solve a problem with de 'emacs-zmq.so' module in my PC)
Thank you for all your help and also for your recent series of videos.
Regards,
Marcos Bujosa
mbujosab
commented
5 months ago Sorry I missed the file I run:
https://github.com/jkitchin/scimax/blob/master/scimax-jupyter-r.org
jkitchin
commented
5 months ago You probably have to get ox-ipynb from https://github.com/jkitchin/ox-ipynb. I don't think it auto installs from anywhere. The rest of those look like use-package failed to download packages from elpa/melpa. The last error about zmq might mean you have to build that library, eg cd to /home/marcos/.emacs.d/elpa/zmq-20230608.1856, and check the makefile to see if you can build it manually.
mbujosab
commented
4 months ago Dear Professor Kitchin,
I was using a rather old Linux distribution, which caused most libraries and Emacs itself to be outdated. I decided to install the latest Debian distribution, update Emacs to version 29, and install the latest version of Scimax. Now, Scimax runs R code without any issues, but it does not export it properly to a Jupyter notebook. I receive an error message that says:
"TypeError: pop expected at most 1 argument, got 2."
Additionally, in the old distribution, I didn't use virtual environments in Python. Now, it seems that I can only install certain libraries if I do so in a virtual environment. As a result, notebooks that previously worked with Scimax no longer function because I don't know how to direct Scimax to use the virtual environment's kernel. I have looked at old issues, but I haven't understood how they were resolved.
Thank you for all your help and for writing scimax. It is great! Best regards,
Marcos
jkitchin
commented
4 months ago I pushed some updates to ox-ipynb that will probably fix the export issue (https://github.com/jkitchin/ox-ipynb/commit/32fcdca5e73acc4efce3469e73124228588df0ab)
I don't use virtual environments. There is no reason you have to use them, you should always be able to install packages in your main environment. (a lot of packages recommend venvs, and some consider it good practice. It might even be necessary for some people who need incompatible versions in different projects. but if you don't need that, I recommend you not use them). This usually leads to https://xkcd.com/1987/ eventually, but my experience with venvs has been almost the same, it is hard to keep track of what should be getting used.
It is a pretty hard problem to get right I think. some venvs modify the path and are meant to be used in a shell, but that doesn't obviously change the path in emacs (I think it gets this from the shell when you start emacs). Maybe if you setup up pyenv, activate the env you want, and then start emacs it will work like you expect. I am not sure about changing envs though.
You might look into https://github.com/marcwebbie/auto-virtualenv
mbujosab
commented
4 months ago Once again, thank you very much for all your help.
The new ox-ipynb update correctly exports the notebook to Jupyter!
However, when exporting the notebook as slides (C-c C-E n s), the browser opens, but the slides are not displayed (although the HTML file is created correctly).
Thank you also for the recommendation about Python environments. I will follow your advice.
jkitchin
commented
4 months ago hm. That is not something I can reproduces, the slideshow works for me at least for this org file: https://github.com/jkitchin/ox-ipynb/blob/master/ipynb-slideshow.org
It also seems to work on an R-notebook for me.
What is the url in the browser?
mbujosab
commented
4 months ago jkitchin
commented
4 months ago I havne't been able to reproduce this. you might see if the command line works, replace %s with the filename.
jupyter nbconvert "%s" --to slides --post serve
mbujosab
commented
4 months ago This is the message in emacs after C-c C-e n s (this succeeds in the conversion to slides)
[NbConvertApp] Converting notebook /home/marcos/CloudStation/ReposGH/Docencia/EconometriaAplicada-SRC/ConR.ipynb to slides
/usr/lib/python3/dist-packages/nbformat/init.py:92: MissingIDFieldWarning: Code cell is missing an id field, this will become a hard error in future nbformat versions. You may want to use normalize() on your notebooks before validations (available since nbformat 5.1.4). Previous versions of nbformat are fixing this issue transparently, and will stop doing so in the future.
validate(nb)
[NbConvertApp] Writing 576660 bytes to /home/marcos/CloudStation/ReposGH/Docencia/EconometriaAplicada-SRC/ConR.slides.html
[NbConvertApp] Redirecting reveal.js requests to https://cdnjs.cloudflare.com/ajax/libs/reveal.js/3.5.0
Serving your slides at http://127.0.0.1:8000/ConR.slides.html
Use Control-C to stop this server
And this is what I get after running: jupyter nbconvert ConR.org --to slides --post serve in a bash term (It does not succeed in the conversion):
[NbConvertApp] Converting notebook ConR.org to slides Traceback (most recent call last): File "/usr/lib/python3/dist-packages/nbformat/reader.py", line 18, in parse_json nb_dict = json.loads(s, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/json/init.py", line 346, in loads return _default_decoder.decode(s) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/json/decoder.py", line 337, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/json/decoder.py", line 355, in raw_decode raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/bin/jupyter-nbconvert", line 33, in
mbujosab
commented
4 months ago I just realized that the problem also occurs with Python Notebooks. With the old Debian (and Scimax) installation, the browser opened the HTML file correctly after converting to slides. With the new one, it doesn't.
jkitchin
commented
4 months ago This sounds like something I saw recently from multiple versions of ox-ipynb being installed. When I run off of the github version it seems to work. i have made a few minor updates since yesterday that might fix some things. Could you update and try again? thx.
mbujosab
commented
4 months ago When exporting to Jupyter notebook I get this from the navigator:
Notebook validation failed: data.cells[2] must be valid exactly by one definition (0 matches found):
"
When exporting to slides I get this in Scimax:
[NbConvertApp] Converting notebook /home/marcos/CloudStation/ReposGH/Docencia/EconometriaAplicada-SRC/Lecc01.ipynb to slides [NbConvertApp] ERROR | Notebook JSON is invalid: data.cells[2] must be valid exactly by one definition (0 matches found)
Failed validating
On instance:
mbujosab
commented
4 months ago This is the output from the terminal when I run
jupyter nbconvert "Lecc01.org" --to slides --post serve
[NbConvertApp] Converting notebook Lecc01.org to slides Traceback (most recent call last): File "/usr/lib/python3/dist-packages/nbformat/reader.py", line 18, in parse_json nb_dict = json.loads(s, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/json/init.py", line 346, in loads return _default_decoder.decode(s) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/json/decoder.py", line 337, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/json/decoder.py", line 355, in raw_decode raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/bin/jupyter-nbconvert", line 33, in
jkitchin
commented
4 months ago That is not an error I have seen before. Can you send me a small org-file that does this and the ipynb it produced?
I am traveling until next Tue, and can take a look at it when I get back.
mbujosab
commented
4 months ago Of course... Example.zip ... and thank you again.
jkitchin
commented
4 months ago When I export your org file to slides, this is what launches. I am not sure why it doesn't work for you.
I am using a pretty current version of Python (3.11) and packages. I don't know if that has anything to do with it.
mbujosab
commented
4 months ago jkitchin
commented
4 months ago For the AnotherExample I saw these things:
jupyter nbconvert LE03.ipynb --to slides --post serve I see this in my browser:
That makes me think the problem is on the nbconvert side. I am using nbconvert 7.16.4 I think, and you have 6.5.3.
Can you see if you can open the ipynb in jupyter fine, e.g.
jupyter lab LE03.ipynb
If that opens with no error, then I guess it is an nbconvert issue. if there is an error, maybe the error is deeper in jupyter.
Also make sure that C-c C-e no makes and opens a notebook without error.
Another thing I noticed in the ipynb is the kernel was set to Python3. This happened probably because the first cell is elisp, so the language defaults to Python. You need to add this to your org file to fix it:
#+OX-IPYNB-LANGUAGE: jupyter-RThank you very much again.
I have installed JupyterLab to do the checks you suggested. But now Scimax has stopped exporting the notebooks to Jupyter. When I try, I get the following error message in Emacs:
Traceback (most recent call last): File "/usr/lib/python3/dist-packages/notebook/traittypes.py", line 235, in _resolve_classes klass = self._resolve_string(klass) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/marcos/.local/lib/python3.11/site-packages/traitlets/traitlets.py", line 2015, in _resolve_string return import_item(string) ^^^^^^^^^^^^^^^^^^^ File "/home/marcos/.local/lib/python3.11/site-packages/traitlets/utils/importstring.py", line 33, in import_item module = import(package, fromlist=[obj]) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ModuleNotFoundError: No module named 'jupyter_server.contents'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/bin/jupyter-notebook", line 33, in
On the other hand, before installing JupyterLab, when I last exported one of the notebooks and got the message "Notebook validation failed," I tried marking one of the blocks as a slide inside the browser, then saved the file from the browser, and went back to Scimax. From Scimax, I tried to create the slides, and it did so without errors. 2ipynbFilesSavedFromScimaxAndFirefox.zip
mbujosab
commented
4 months ago Ah!... and it seems that JupyterLab opens all the .ipynb files (but at this moment I can't test trying to generate an .ipynb from Scimax, see that I get the warning message, and then try to open it from JupyterLab).
jkitchin
commented
4 months ago Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/notebook/traittypes.py", line 235, in _resolve_classesis an error on the Python/Jupyter side. It either means ox-ipynb made a bad notebook, or the jupyter installation is not correct somehow.
there are a series of steps you can test to see where the problem lies. The first step is to try
M-x ox-ipynb-export-to-ipynb-buffer
this will show you the json that gets created. If this works without error then it means ox-ipynb works fine probably.
Next,
M-x ox-ipynb-export-to-ipynb-file
that saves the output to a file. You can check to see if you can open that file in a shell. All ox-ipynb does is call shell command like this "jupyter notebook file.ipynb"
For slides, all that happens is the org file is turned into an ipynb using the commands above, and then a shell command runs like jupyter nbconvert fname.ipynb --to slides --post serve
I think I see what is wrong in your scimax version. There are cells like
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": null
}
],The null should not be there. Can you send me the org-file that generated this ipynb so I can look into how that happened?
mbujosab
commented
4 months ago Both the command "M-x ox-ipynb-export-to-ipynb-buffer" and the command "M-x ox-ipynb-export-to-ipynb-file" have worked without errors. Unfortunately, since I installed JupyterLab, the jupyter command doesn't work. When I execute "jupyter notebook file.ipynb," I get an error:
jupyter notebook Lecc03.ipynb Traceback (most recent call last): File "/usr/lib/python3/dist-packages/notebook/traittypes.py", line 235, in _resolve_classes klass = self._resolve_string(klass) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/marcos/.local/lib/python3.11/site-packages/traitlets/traitlets.py", line 2015, in _resolve_string return import_item(string) ^^^^^^^^^^^^^^^^^^^ File "/home/marcos/.local/lib/python3.11/site-packages/traitlets/utils/importstring.py", line 33, in import_item module = import(package, fromlist=[obj]) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ModuleNotFoundError: No module named 'jupyter_server.contents'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/bin/jupyter-notebook", line 33, in
jkitchin
commented
4 months ago something is not right in the jupyter installation, maybe it is version incompatibility.? This in particular ModuleNotFoundError: No module named 'jupyter_server.contents' indicates something is not completely installed.
I guess you get an error just running jupyter notebook?
Here are the settings I have that seem to work.
#+BEGIN_SRC sh
jupyter --version
#+END_SRC
#+RESULTS:
| Selected | Jupyter | core | packages... |
| IPython | : | 8.26.0 | |
| ipykernel | : | 6.29.5 | |
| ipywidgets | : | not | installed |
| jupyter_client | : | 8.6.2 | |
| jupyter_core | : | 5.7.2 | |
| jupyter_server | : | 2.14.2 | |
| jupyterlab | : | not | installed |
| nbclient | : | 0.10.0 | |
| nbconvert | : | 7.16.4 | |
| nbformat | : | 5.10.4 | |
| notebook | : | not | installed |
| qtconsole | : | not | installed |
| traitlets | : | 5.14.3 | |
#+BEGIN_SRC sh
python --version
#+END_SRC
#+RESULTS:
: Python 3.11.4
This is what I get
jupyter --version
| Selected | Jupyter | core | packages... | | IPython | : | 8.5.0 | | | ipykernel | : | 6.17.0 | | | ipywidgets | : | 6.0.0 | | | jupyter_client | : | 7.4.9 | | | jupyter_core | : | 4.12.0 | | | jupyter_server | : | 2.14.2 | | | jupyterlab | : | 4.2.3 | | | nbclient | : | 0.7.2 | | | nbconvert | : | 6.5.3 | | | nbformat | : | 5.5.0 | | | notebook | : | 6.4.12 | | | qtconsole | : | not | installed | | traitlets | : | 5.14.3 | |
python3 --version
: Python 3.11.2
mbujosab
commented
4 months ago With this
pip3 install --user notebook==6.5.6 --break-system-packages
taken from this https://stackoverflow.com/questions/77549493/modulenotfounderror-no-module-named-jupyter-server-contents now jupyter notebook runs.
So now:
jkitchin
commented
4 months ago I tried adding some logic in 352303a6 that might help. I think the problem is you are getting text: null entries, but that should not be possible I think (yet you see them, and I can't reproduce them). I guess that saving it must fix the issue, but I don't understand how you get it open in the first place if the error happens.
mbujosab
commented
4 months ago I am traveling until next Sunday. I will explore a little more next week. Thank you again for your help. Marcos.
mbujosab
commented
4 months ago Hello again,
The problem arises when including as a header in a code block ":results silent :exports code".
In that case, the .ipynb file is generated, but when opened, Jupyter indicates that the notebook is invalid, and then it is not possible to generate the slides from the .ipynb file.
I am attaching two minimal examples. The difference between them is that in one, the header ":results silent :exports code" is specified, and in the other, it is not.
I am also attaching the .ipynb files generated in both cases.
(When the invalid file is opened in the browser and saved to the disk from the browser, the saved .ipynb file is somehow corrected. However, the one initially generated from scimax is invalid).
jkitchin
commented
4 months ago Does that happen with the latest version of ox-ipynb? I am not able to reproduce these outputs on my end.
The problem is definitely this in the one that doesn't work:
"text": null
but that should not be possible with the new version I think.
mbujosab
commented
4 months ago I was using the version of ox-ipynb.el from the commit of July 11th. I just tried the latest version, and it seems that everything works fine now. Once again, thank you very much for your help and for creating Scimax. It’s great.
jkitchin
commented
4 months ago Thanks for working through all the issues with me. I am glad we got it fixed!
Hello,
I am wondering if there are any users who have successfully created Jupyter notebooks with R code using Scimax. I have attempted to do so, but I have not been able to get it to work.
Any guidance or suggestions would be greatly appreciated.
Thank you!
Best regards,
Marcos Bujosa