search

quarto-dev / quarto-cli

Open-source scientific and technical publishing system built on Pandoc.
https://quarto.org
Other
3.75k stars 306 forks source link

KeyError: 'kernel_dirs' - while running quarto as singularity #10499

Closed bhagesh-h closed 2 weeks ago

bhagesh-h commented 1 month ago

Bug description

I am trying to run quarto via singularity, and received KeyError: 'kernel_dirs' when I run a quarto check and the same while running quarto render.

please find complete error below

Steps to reproduce

_metadata.yaml content

format:
  html:
    toc-title: |
      ![](images/logo.png)

      Contents
    css: css/styles.css
logo: images/logo.png
csl: csl/Diabetologia.csl
bibliography: references/ref.bib

report.qmd content

---
title: ' '
date: today
format:
  html:
    toc: true
    toc-location: left
    code-fold: true
    smooth-scroll: true
    self-contained: true
    number-sections: true
    anchor-sections: true
    embed-resources: true
jupyter: python3
filters:
   - /ext/quarto/_extensions/quarto-ext/include-code-files/include-code-files.lua
execute:
  freeze: false
  echo: false
  cache: false
---

```{python}
#| tags: [parameters]

version="Placeholder version"
project_ID="Placeholder project_ID"
pipeline="Placeholder pipeline name"
read_merging_metrics="Placeholder read_merging_metrics"
sequence_cleaning_metrics="Placeholder sequence_cleaning_metrics"

::: callout-note

Project Details

Project ID: {python} project_id\ Date of Processing: {{< meta date >}}\ Pipeline: {python} pipeline\ Version: {python} version :::

References {.unnumbered}

::: {#refs} :::

{.unnumbered .unlisted}

SOME TEXT {.unnumbered .unlisted .center}

# Singularity command
```shell
singularity shell --bind /mnt/,/mnt/temp:/run/user/ quarto_v1_5_56.sif

quarto commands

export XDG_RUNTIME_DIR='/mnt/result/'
export QUARTO_PRINT_STACK=true

quarto check

quarto render /mnt/result/report.qmd \
--to html --output-dir quarto \
--metadata-file /mnt/result/_metadata.yaml \
--metadata title="Test Report" \
-P pipeline="Test Pipeline" \
-P project_id=12345 \
-P version=v1.1.1 \
-P sequence_cleaning_metrics=/mnt/result/sequence_cleaning_metrics.tsv \
-P read_merging_metrics=/mnt/result/read_merging_metrics.tsv

Expected behavior

No response

Actual behavior

No response

Your environment

Quarto check output

Error:

Quarto 1.5.56

[✓] Checking versions of quarto binary dependencies...
      Pandoc version 3.2.0: OK
      Dart Sass version 1.70.0: OK
      Deno version 1.41.0: OK
      Typst version 0.11.0: OK

[✓] Checking versions of quarto dependencies......OK

[✓] Checking Quarto installation......OK
      Version: 1.5.56
      Path: /opt/quarto/bin

[✓] Checking tools....................OK
      TinyTeX: (not installed)
      Chromium: (not installed)

[✓] Checking LaTeX....................OK
      Tex:  (not detected)

[✓] Checking basic markdown render....OK

[✓] Checking Python 3 installation....OK
      Version: 3.12.2 (Conda)
      Path: /opt/conda/bin/python
      Jupyter: 5.7.2
      Kernels: python3

(\) Checking Jupyter engine render....2024-08-12 13:18:21,047 - traitlets - ERROR - /opt/conda/lib/python3.12/lib-dynload/_sqlite3.cpython-312-x86_64-linux-gnu.so: undefined symbol: sqlite3_deserialize
Traceback (most recent call last):
  File "/opt/conda/lib/python3.12/site-packages/traitlets/traitlets.py", line 632, in get
    value = obj._trait_values[self.name]
            ~~~~~~~~~~~~~~~~~^^^^^^^^^^^
KeyError: 'kernel_dirs'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/conda/lib/python3.12/site-packages/jupyter_client/manager.py", line 87, in wrapper
    out = await method(self, *args, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.12/site-packages/jupyter_client/manager.py", line 435, in _async_start_kernel
    kernel_cmd, kw = await self._async_pre_start_kernel(**kw)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.12/site-packages/jupyter_client/manager.py", line 397, in _async_pre_start_kernel
    self.kernel_spec,
    ^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.12/site-packages/jupyter_client/manager.py", line 195, in kernel_spec
    self._kernel_spec = self.kernel_spec_manager.get_kernel_spec(self.kernel_name)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.12/site-packages/jupyter_client/kernelspec.py", line 282, in get_kernel_spec
    resource_dir = self._find_spec_directory(kernel_name.lower())
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.12/site-packages/jupyter_client/kernelspec.py", line 256, in _find_spec_directory
    for kernel_dir in [kd for kd in self.kernel_dirs if os.path.isdir(kd)]:
                                    ^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.12/site-packages/traitlets/traitlets.py", line 687, in __get__
    return t.cast(G, self.get(obj, cls))  # the G should encode the Optional
                     ^^^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.12/site-packages/traitlets/traitlets.py", line 635, in get
    default = obj.trait_defaults(self.name)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.12/site-packages/traitlets/traitlets.py", line 1897, in trait_defaults
    return t.cast(Sentinel, self._get_trait_default_generator(names[0])(self))
                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.12/site-packages/jupyter_client/kernelspec.py", line 195, in _kernel_dirs_default
    from IPython.paths import get_ipython_dir
  File "/opt/conda/lib/python3.12/site-packages/IPython/__init__.py", line 55, in <module>
    from .terminal.embed import embed
  File "/opt/conda/lib/python3.12/site-packages/IPython/terminal/embed.py", line 15, in <module>
    from IPython.core.interactiveshell import DummyMod, InteractiveShell
  File "/opt/conda/lib/python3.12/site-packages/IPython/core/interactiveshell.py", line 110, in <module>
    from IPython.core.history import HistoryManager
  File "/opt/conda/lib/python3.12/site-packages/IPython/core/history.py", line 10, in <module>
    import sqlite3
  File "/opt/conda/lib/python3.12/sqlite3/__init__.py", line 57, in <module>
    from sqlite3.dbapi2 import *
  File "/opt/conda/lib/python3.12/sqlite3/dbapi2.py", line 27, in <module>
    from _sqlite3 import *
ImportError: /opt/conda/lib/python3.12/lib-dynload/_sqlite3.cpython-312-x86_64-linux-gnu.so: undefined symbol: sqlite3_deserialize

/opt/conda/lib/python3.12/lib-dynload/_sqlite3.cpython-312-x86_64-linux-gnu.so: undefined symbol: sqlite3_deserialize

[✓] Checking Jupyter engine render....OK

ERROR: Error
    at renderFiles (file:///opt/quarto/bin/quarto.js:78081:29)
    at eventLoopTick (ext:core/01_core.js:153:7)
    at async render (file:///opt/quarto/bin/quarto.js:82929:21)
    at async checkJupyterRender (file:///opt/quarto/bin/quarto.js:101297:21)
    at async file:///opt/quarto/bin/quarto.js:101261:21
    at async withSpinner (file:///opt/quarto/bin/quarto.js:76250:16)
    at async checkJupyterInstallation (file:///opt/quarto/bin/quarto.js:101257:17)
    at async check (file:///opt/quarto/bin/quarto.js:101072:13)
    at async Command.actionHandler (file:///opt/quarto/bin/quarto.js:101389:5)
    at async Command.execute (file:///opt/quarto/bin/quarto.js:8017:13)
mcanouil commented 1 month ago

Thanks for the report

Thanks for report. Unfortunately, as is, we can't use it.

Note that we are asking a reproducible example because without it's very unlikely we'll be able to guess what is wrong. For instance, you did not provide the content of the Quarto document. You shared the quarto check output for Quarto 1.5.56 but the sif image displays quarto_v1_4_553.sif. Also, when trying to debug, you should try a small and simple document which is not the case based on the Quarto command you shared (see examples below).

Could you share a small self-contained "working" (reproducible) example to work with, i.e., a complete Quarto document or a Git repository? The goal is to make it as easy as possible for us to recreate your problem so that we can fix it: please help us help you! Thanks.

You can share a self-contained "working" (reproducible) Quarto document using the following syntax, i.e., using more backticks than you have in your document (usually four ````). See https://quarto.org/bug-reports.html#small-is-beautiful-aim-for-a-single-document-with-10-lines.

If you have multiple files (and if it is absolutely required to have multiple files), please share as a Git repository.

RPython
`````md ````qmd --- title: "Reproducible Quarto Document" format: html engine: knitr --- This is a reproducible Quarto document. {{< lipsum 1 >}} ```{r} x <- c(1, 2, 3, 4, 5) y <- c(1, 4, 9, 16, 25) plot(x, y) ``` ![An image]({{< placeholder 600 400 >}}){#fig-placeholder} {{< lipsum 1 >}} The end after @fig-placeholder. ```` ````` `````md ````qmd --- title: "Reproducible Quarto Document" format: html engine: jupyter --- This is a reproducible Quarto document. {{< lipsum 1 >}} ```{python} import matplotlib.pyplot as plt x = [1, 2, 3, 4, 5] y = [1, 4, 9, 16, 25] plt.plot(x, y) plt.show() ``` ![An image]({{< placeholder 600 400 >}}){#fig-placeholder} {{< lipsum 1 >}} The end after @fig-placeholder. ```` `````

Additionally and if not already given, please share the output of quarto check within a code blocks (i.e., using three backticks ```txt), see https://quarto.org/bug-reports.html#check.

mcanouil commented 1 month ago

For reference and information:

bhagesh-h commented 1 month ago

thank you @mcanouil , I have updated the query above with examples and corrected singularity version.

github-actions[bot] commented 2 weeks ago

Thank you for using Quarto and reporting an issue!

Unfortunately, this issue is now considered stale because it has been opened since 14 days without providing a "working" reproducible example to help us investigate. If you are still facing the issue, please review the "Bug Reports" guide on how to provide a fully reproducible example as a self-contained Quarto document or a link to a Git repository. Without a reproducible example, it is unlikely that the issue will be addressed.

You can share a Quarto document using the following syntax, i.e., using more backticks than you have in your document (usually four ````).

````qmd
---
title: "Reproducible Quarto Document"
format: html
---

This is a reproducible Quarto document using `format: html`.
It is written in Markdown and contains embedded R code.
When you run the code, it will produce a plot.

```{r}
plot(cars)

The end.

cscheid commented 2 weeks ago

Unfortunately, we can't really help you here, since we're not singularity users nor do we have the time to learn it.