search

quarto-dev / quarto-cli

Open-source scientific and technical publishing system built on Pandoc.
https://quarto.org
Other
3.77k stars 308 forks source link

quarto always uses a completely removed Jupyter kernel #9438

Closed xwuupb closed 5 months ago

xwuupb commented 5 months ago

Bug description

quarto always uses a completely removed Jupyter kernel.

I installed Julia version 1.10 and everything looks well-configured.

$ jupyter-kernelspec list
Available kernels:
  julia-1.10    /home/xxx/.local/share/jupyter/kernels/julia-1.10
  python3       /usr/share/jupyter/kernels/python3

$ quarto check jupyter

[✓] Checking Python 3 installation....OK
      Version: 3.6.9
      Path: /usr/bin/python3
      Jupyter: 4.9.2
      Kernels: julia-1.10, python3

[✓] Checking Jupyter engine render....OK

However, quarto always uses a completely removed Jupyter kernel, when I

$ quarto render a.qmd
ERROR: Jupyter kernel 'julia-1.8' not found.

Steps to reproduce

No response

Expected behavior

quarto should not use the removed Jupyter kernel.

Actual behavior

quarto uses the removed Jupyter kernel.

Your environment

I do not think this is OS-relevant

Quarto check output

see above

mcanouil commented 5 months ago

Thanks for the report.

I do not think this is OS-relevant

Let us be judge of that. Your environment matters, please provide your OS and IDE if you are using one.

Also, could you check your home directory to ensure you don't have an old Julia installation. The issue is that Julia can come with a Conda environment which contains Python/Jupyter. If this conda environment is activated, it might confuse Quarto.

Finally, it's important to provide the complete output of quarto check not the subset you think is relevant. For instance, we need the version of Quarto you used.

xwuupb commented 5 months ago

Thanks for your very fast response!

could you check your home directory to ensure you don't have an old Julia installation.

All old Julia installations are completely removed. I have verified this via $PATH.

If this conda environment is activated, it might confuse Quarto.

Thanks and I will double check from my side.

Finally, it's important to provide the complete output of quarto check not the subset you think is relevant. For instance, we need the version of Quarto you used.

Thanks and see below:

$ dpkg-query -l | grep -i quarto; quarto --version; quarto check
ii  quarto                                                                  1.4.553                                                         amd64        Quarto is an academic, scientific, and technical publishing system built on Pandoc.
1.4.553
Quarto 1.4.553
[✓] Checking versions of quarto binary dependencies...
      Pandoc version 3.1.11: OK
      Dart Sass version 1.69.5: OK
      Deno version 1.37.2: OK
[✓] Checking versions of quarto dependencies......OK
[✓] Checking Quarto installation......OK
      Version: 1.4.553
      Path: /opt/quarto/bin

[✓] Checking tools....................OK
      TinyTeX: (not installed)
      Chromium: (not installed)

[✓] Checking LaTeX....................OK
      Using: Installation From Path
      Path: /usr/local/texlive/2022/bin/x86_64-linux
      Version: 2022

[✓] Checking basic markdown render....OK

[✓] Checking Python 3 installation....OK
      Version: 3.6.9
      Path: /usr/bin/python3
      Jupyter: 4.9.2
      Kernels: julia-1.10, python3

(/) Checking Jupyter engine render....

module 'asyncio' has no attribute 'get_running_loop'
sys:1: RuntimeWarning: coroutine 'get_info' was never awaited
(-) Checking Jupyter engine render....[IPKernelApp] WARNING | Parent appears to have exited, shutting down.
[✓] Checking Jupyter engine render....OK

I will also continue my local trouble-shooting.

cscheid commented 5 months ago

It's not Quarto. Jupyter maintains a global list of installed kernels, and all that Quarto can do is query it with "jupyter kernelspec list". Run that command yourself and if you see a path to a removed kernel, you'll need to do the removal yourself. In my opinion, that's a badly designed feature in Jupyter, but it's not one we can avoid.

xwuupb commented 5 months ago

all that Quarto can do is query it with "jupyter kernelspec list".

In my 1st post I show that

$ jupyter-kernelspec list
Available kernels:
  julia-1.10    /home/xxx/.local/share/jupyter/kernels/julia-1.10
  python3       /usr/share/jupyter/kernels/python3

When I run quarto render a.qmd, quarto tries to use the julia-1.8 kernel, which is not listed above, see

$ quarto render a.qmd
ERROR: Jupyter kernel 'julia-1.8' not found.

Thus, this is indeed a bug in quarto.

xwuupb commented 5 months ago

@cscheid Could you please read my last post carefully, re-open this issue? As the error shows, it is a bug in quarto.

mcanouil commented 5 months ago

Could you try:

jupyter kernelspec list
xwuupb commented 5 months ago

Could you try: ...

See my post here: https://github.com/quarto-dev/quarto-cli/issues/9438#issue-2255066929

mcanouil commented 5 months ago

@xwuupb You are not using the same command (even if it outputs the same). Also Quarto cannot detect something that does not exist, meaning there is a julia-1.8 kernel somewhere on your machine.

Check manually you don't have a kernel or wrong setup, go to /home/xxx/.local/share/jupyter/kernels/julia-1.10 and check the files, especially the kernel.json.

xwuupb commented 5 months ago

especially the kernel.json

Thanks for your hints and it looks like:

$ cat kernel.json
{
  "display_name": "Julia 1.10.2",
  "argv": [
    "/home/xxx/bin/julia-1.10.2/bin/julia",
    "-i",
    "--color=yes",
    "--project=@.",
    "/home/xxx/.julia-pkg/packages/IJulia/Vo51o/src/kernel.jl",
    "{connection_file}"
  ],
  "language": "julia",
  "env": {},
  "interrupt_mode": "signal"
}

I think it's a bug in Quarto. I would also like to troubleshoot this problem.

mcanouil commented 5 months ago

What document front matter or _quarto.yml are you using exactly?

Again, no kernels are hardcoded in Quarto. If Quarto sees a 1.8, it comes from your settings/environment.

cscheid commented 5 months ago

jupyter-kernelspec list is not the same as jupyter kernelspec list. The latter is the only way that Quarto determines and uses engines.

It's also possible that you have two separate versions of jupyter, and that the one that quarto check finds is not the one that you're using when you're listing it. If Quarto is using a version of a kernel that you uninstalled, that is happening because some installation of Jupyter in your system is reporting it to Quarto. That's literally all that Quarto does and can do. The code is here: https://github.com/quarto-dev/quarto-cli/blob/main/src/core/jupyter/kernels.ts#L79

xwuupb commented 5 months ago

The "bug" of ERROR: Jupyter kernel 'julia-1.8' not found. is finally identified to be caused by the line jupyter: julia-1.8 in this out-of-date example here: https://quarto.org/docs/computations/julia.html#code-blocks (maybe I'm the only naive user, who did a simple copy-paste-run workflow).

mcanouil commented 5 months ago

The line is not outdated. The example is still a working example if you have Julia 1.8 or if you install Julia 1.8.

Code should never be copied and pasted without reading and trying to understand it first.

It seems obvious enough that setting jupyter: julia-1.8 means to use Julia 1.8 and not something else. There is no reason to update the example based on Julia releases. Even if the example was a generic one such as jupyter: julia-X.X.X, you would have seen the error message.