JSON readtable error

[DiogoFerrari]

ob-ipython is great and works well, it is breaking other functionalities. It may be too vague, but weirdly enough, after loading ob-ipython, I cannot export other org files to pdf because it gives that "JSON readtable error", even though the .org doesn't have any code block.

I have no clue where to start looking at to find why ob-ipython is breaking other functionalities. The debug messages are not helping very much (considering the short time I have to invest in fixing it myself at the moment). Any help is much appreciated.

The problem only happens when I load emacs and add ipython to org-babel-load-langugage with

(custom-set-variables '(org-babel-load-languages '((ipython . t))))

[DiogoFerrari]

One situation the "JSON readtable error" happens is when I include another org file with:

#+INCLUDE: <another-org-file>.org

[dcherian]

Use emacs-jupyter: https://github.com/dzop/emacs-jupyter

[DiogoFerrari]

It requires emacs 26. I will try that later. Thanks for the suggestion.

[DiogoFerrari]

Anyone?

[DiogoFerrari]

I found the solution. The problem is the function

shell-command-to-string

of this function:

(defun ob-ipython--get-kernels ()
  "Return a list of available jupyter kernels and their corresponding languages.
The elements of the list have the form (\"kernel\" \"language\")."
  (and ob-ipython-command
       (let ((kernelspecs (cdar (json-read-from-string
                                 (shell-command-to-string
                                  (s-concat ob-ipython-command " kernelspec list --json"))))))
         (-map (lambda (spec)
                 (cons (symbol-name (car spec))
                       (->> (cdr spec)
                            (assoc 'spec)
                            cdr
                            (assoc 'language)
                            cdr)))
               kernelspecs))))

In my .emacs.d I had

(setq shell-command-switch "-ic")

The option -i resulted in an error message from shell interactive command:

bash: cannot set terminal process group (-1): Inappropriate ioctl for device
bash: no job control in this shell

The function ob-ipython--get-kernels above is concatenating that error message to the json string, resulting in the error "JSON readtable error".

Solution 1: use just

(setq shell-command-switch "-c")

Solution 2 (better): change shell-command-to-string in that function to make sure the output does not include any error or warning message from the shell. I will probably do that later, but maybe someone here knows a quick fix in that direction.

[jkitchin]

FWIW the shell-command-switch variable setting of "-ic" does not cause this problem for me. I am not sure why. I am using a Mac. A quick solution would be to let-bind shell-command-switch inside the function, e.g.

(defun ob-ipython--get-kernels ()
  "Return a list of available jupyter kernels and their corresponding languages.
The elements of the list have the form (\"kernel\" \"language\")."
  (and ob-ipython-command
       (let* ((shell-command-switch "-c")
          (kernelspecs (cdar (json-read-from-string
                  (shell-command-to-string
                   (s-concat ob-ipython-command " kernelspec list --json"))))))
         (-map (lambda (spec)
                 (cons (symbol-name (car spec))
                       (->> (cdr spec)
                            (assoc 'spec)
                            cdr
                            (assoc 'language)
                            cdr)))
               kernelspecs))))

[Tass0sm]

This also happens if the command from jupyter gives any error. Mine was complaining about the presence of both ~/.config/ipython and ~/.ipython. After "~/.config/ipython" was deleted, ob-ipython--get-kernels could run successfully.

When this is broken, it also breaks other aspects of org, like captures, for some reason.

[mandarm]

Solution 2 (better): change shell-command-to-string in that function to make sure the output does not include any error or warning message from the shell. I will probably do that later, but maybe someone here knows a quick fix in that direction.

If I understand your idea correctly, using the following instead of (shell-command-to-string some-command) might solve the problem:

(with-output-to-string
    (with-current-buffer
      standard-output
      (process-file shell-file-name nil '(t nil)  nil shell-command-switch some-command)))

See: https://emacs.stackexchange.com/questions/21422/how-to-discard-stderr-when-running-a-shell-function

Background: using Python 3.11 creates a similar problem. The command

python -Xfrozen_modules=off -m jupyter kernelspec list --json 

outputs the following to stderr:

0.00s - Debugger warning: It seems that frozen modules are being used, which may
0.00s - make the debugger miss breakpoints. Please pass -Xfrozen_modules=off
0.00s - to python to disable frozen modules.
0.00s - Note: Debugging will proceed. Set PYDEVD_DISABLE_FILE_VALIDATION=1 to disable this validation.

And this throws ob-ipython off.