quarto-dev / quarto-cli

Open-source scientific and technical publishing system built on Pandoc.
https://quarto.org
Other
3.77k stars 309 forks source link

passing multiple QUARTO_DENO_EXTRA_OPTIONS flags #9607

Closed knaaptime closed 4 months ago

knaaptime commented 4 months ago

Bug description

thanks for all your work on this! I'm hitting an issue where I need to pass multiple flags to the QUARTO_DENO_EXTRA_OPTIONS environment variable, but i only seem to be able to pass one at a time.

Steps to reproduce

Specifically, when trying to build this project (I can open-source it shortly if for some reason you want to try building), I need to use export QUARTO_DENO_EXTRA_OPTIONS=--v8-flags=--stack-size=8176 (following the advice here, otherwise the build will fail when trying to generate this notebook

But I also need to include export QUARTO_DENO_EXTRA_OPTIONS="--v8-flags=--max-old-space-size=10000", otherwise the render crashes at the very end of the build. In this case, the pages themselves all get generated correctly, but the final pieces (moving references to the designated section and generating the sitemap and search functionality, etc) all fail out

Expected behavior

So, an ideal solution would be to pass both flags to the QUARTO_DENO_EXTRA_OPTIONS variable, but I cant seem to do that... Any space between the flags causes the parsing to get screwed up once the environment variables are subbed in, and deno refuses to accept more than one --v8-flags= option. Any ideas?

maybe the call "${QUARTO_DENO}" ${QUARTO_ACTION} ${QUARTO_DENO_OPTIONS} ${QUARTO_DENO_EXTRA_OPTIONS} "${QUARTO_IMPORT_MAP_ARG}" "${QUARTO_TARGET}" "$@" could just be modified to iterate through a list of QUARTO_DENO_EXTRA_OPTIONS if they exist?

Actual behavior

If the first flag is omitted, the build fails with

ERROR: RangeError: Maximum call stack size exceeded

Stack trace:
    at countTicks (file:///Applications/quarto/bin/quarto.js:31848:17)
    at ticksForCode (file:///Applications/quarto/bin/quarto.js:31851:27)
    at mdFormatOutput (file:///Applications/quarto/bin/quarto.js:32301:19)
    at mdHtmlOutput (file:///Applications/quarto/bin/quarto.js:32949:12)
    at mdOutputDisplayData (file:///Applications/quarto/bin/quarto.js:32845:20)
    at mdFromCodeCell (file:///Applications/quarto/bin/quarto.js:32735:35)
    at jupyterToMarkdown (file:///Applications/quarto/bin/quarto.js:32156:34)
    at eventLoopTick (ext:core/01_core.js:183:11)
    at async Object.execute (file:///Applications/quarto/bin/quarto.js:38392:25)
    at async renderExecute (file:///Applications/quarto/bin/quarto.js:77017:27)

if the second flag is omitted, the render appears to work, but crashes at the end with

<--- Last few GCs --->

[12059:0x110018000]   261671 ms: Scavenge (reduce) 1314.5 (1320.4) -> 1314.5 (1320.4) MB, 0.25 / 0.00 ms  (average mu = 0.999, current mu = 0.986) allocation failure; 
[12059:0x110018000]   261727 ms: Mark-Compact (reduce) 1451.2 (1457.1) -> 1451.2 (1455.4) MB, 12.38 / 0.00 ms  (average mu = 0.996, current mu = 0.895) allocation failure; scavenge might not succeed

<--- JS stacktrace --->

#
# Fatal JavaScript out of memory: Reached heap limit
#

/Applications/quarto/bin/quarto: line 177: 12059 Trace/BPT trap: 5       "${QUARTO_DENO}" ${QUARTO_ACTION} ${QUARTO_DENO_OPTIONS} ${QUARTO_DENO_EXTRA_OPTIONS} "${QUARTO_IMPORT_MAP_ARG}" "${QUARTO_TARGET}" "$@"

Your environment

macos 14.4.1 rendering in a terminal

Quarto check output

Quarto 1.4.554 [✓] Checking versions of quarto binary dependencies... Pandoc version 3.1.11: OK Dart Sass version 1.69.5: OK Deno version 1.37.2: OK [✓] Checking versions of quarto dependencies......OK [✓] Checking Quarto installation......OK Version: 1.4.554 Path: /Applications/quarto/bin

[✓] Checking tools....................OK TinyTeX: (not installed) Chromium: (not installed)

[✓] Checking LaTeX....................OK Using: Installation From Path Path: /Library/TeX/texbin Version: 2024

[✓] Checking basic markdown render....OK

[✓] Checking Python 3 installation....OK Version: 3.10.8 (Conda) Path: /Users/knaaptime/mambaforge/bin/python Jupyter: 5.1.3 Kernels: conda-env-thompson-trajectories-py, conda-env-libpysal-py, conda-env-workshop-pysal-wrsa24-py, conda-env-urban_analysis-py, conda-env-carto_sdsc23-py, conda-env-access-py, conda-env-pointpats-py, conda-env-seg_networks-py, conda-env-thompson-discrete-choice-py, conda-env-paper-geographs-narsc23-py, conda-env-submarket_boundaries-py, conda-env-esda-py, conda-env-testproj-py, conda-env-rentpanel-py, conda-env-jlab-py, conda-env-pysal-narsc22-py, conda-env-schalk-py, conda-env-knox_opioids-py, conda-env-r-r, conda-env-pysal_carto_sdss-py, conda-env-pysal-iale2022-py, conda-env-md_hedonics-py, conda-env-spreg-py, conda-env-getis_empcenter-py, conda-env-edu_concordance-py, conda-env-riskmodel-py, conda-env-geosnap-py, conda-env-bikeped-py, conda-env-instructed-py, conda-env-pysal-workshop-py, conda-root-py, submarket_boundaries, conda-env-tossme-py, conda-env-test-py, conda-env-carto_sdsc24-py, conda-env-tobler-py, conda-env-workshop-pysal-narsc23-py, conda-env-incseg-py, conda-env-segregation-py, python3

(|) Checking Jupyter engine render....Traceback (most recent call last): File "/Applications/quarto/share/jupyter/jupyter.py", line 21, in from notebook import notebook_execute, RestartKernel File "/Applications/quarto/share/jupyter/notebook.py", line 14, in from yaml import safe_load ModuleNotFoundError: No module named 'yaml' There is a environment.yml file in this directory. Is this for a conda env that you need to restore?

[✓] Checking Jupyter engine render....OK

knaaptime commented 4 months ago

although maybe with the upcoming 1.5 release one flag wont be needed anymore, so I'll just need the one?

cscheid commented 4 months ago

Thanks for the kind words.

QUARTO_DENO_EXTRA_OPTIONS is an internal option that we only describe in the troubleshooting page.

This isn't a Quarto bug - you just unfortunately need to know your way around deno and v8. You need to use the correct syntax:

export QUARTO_DENO_EXTRA_OPTIONS="--v8-flags=--max-old-space-size=10000,--stack-size=8176"
knaaptime commented 4 months ago

sweet, many thanks. I scoured the deno docs yesterday and tried a dozen different combos with different delimiters to include both args--except, apparently a comma with no space :)

thanks again!