quarto-ext / shinylive

Quarto extension to embed Shinylive for Python applications
https://quarto-ext.github.io/shinylive/
MIT License
144 stars 8 forks source link

shinylive on windows apparently not working? #8

Closed cscheid closed 1 year ago

cscheid commented 1 year ago

I can't forward issues across organizations, but take a look at https://github.com/quarto-dev/quarto-cli/issues/4207

snhansen commented 1 year ago

Same issue when trying on Debian (WSL-2 on Windows):

(base) stefan@D42538:~/quartotest$ quarto render test2.qmd
pandoc
  to: html
  output-file: test2.html
  standalone: true
  section-divs: true
  html-math-method: mathjax
  wrap: none
  default-image-extension: png

metadata
  document-css: false
  link-citations: true
  date-format: long
  lang: en
  title: Shinylive in Quarto example

Error running filter /opt/quarto/share/filters/main.lua:
PandocLuaError "PandocFilterError \"/home/stefan/quartotest/shinylive\" \"Could not find executable /home/stefan/quartotest/shinylive\"\nstack traceback:\n\t/opt/quarto/share/filters/main.lua:3929: in function </opt/quarto/share/filters/main.lua:3909>\n\t[C]: in ?\n\t[C]: in method 'walk'\n\t/opt/quarto/share/filters/main.lua:153: in function 'run_emulated_filter'\n\t/opt/quarto/share/filters/main.lua:405: in local 'callback'\n\t/opt/quarto/share/filters/main.lua:410: in upvalue 'run_emulated_filter_chain'\n\t/opt/quarto/share/filters/main.lua:451: in function </opt/quarto/share/filters/main.lua:432>"
stack traceback:
        /opt/quarto/share/filters/main.lua:153: in function 'run_emulated_filter'
        /opt/quarto/share/filters/main.lua:405: in local 'callback'
        /opt/quarto/share/filters/main.lua:410: in upvalue 'run_emulated_filter_chain'
        /opt/quarto/share/filters/main.lua:451: in function </opt/quarto/share/filters/main.lua:432>

This is the result of quarto check on that platform:

(base) stefan@D42538:~/quartotest$ quarto check

[✓] Checking versions of quarto binary dependencies...
      Pandoc version 2.19.2: OK
      Dart Sass version 1.32.8: OK
[✓] Checking versions of quarto dependencies......OK
[✓] Checking Quarto installation......OK
      Version: 1.3.154
      Path: /opt/quarto/bin

[✓] Checking basic markdown render....OK

[✓] Checking Python 3 installation....OK
      Version: 3.9.13 (Conda)
      Path: /home/stefan/anaconda3/bin/python
      Jupyter: 4.11.1
      Kernels: python3

[✓] Checking Jupyter engine render....OK

[✓] Checking R installation...........OK
      Version: 4.0.4
      Path: /usr/lib/R
      LibPaths:
        - /usr/local/lib/R/site-library
        - /usr/lib/R/site-library
        - /usr/lib/R/library
      rmarkdown: 2.20

[✓] Checking Knitr engine render......OK
cderv commented 1 year ago

@snhansen I can't reproduce on WSL 2 ubuntu. I need to setup debian to try like you.

Can you check several things in the mean time :

I am seeing this

"Could not find executable /home/stefan/quartotest/shinylive\"

And not sure why your executable shinylive would be searched there. Is this a known folder or file to you ?

On Windows, it searches in similar folder (D:\\\\quartotest\\\\shinylive\\\\shinylive\). I wonder why. 🤔

I am using 1.3.185 - last prerelease.

snhansen commented 1 year ago

OK, so a little update from here. I was definitely not making sure that the shinylive-extension was installed in the Quarto project directory. So that explains the Could not find executable messages for both instances.

I'm still having issues with the shinylive Python package it seems.

On Windows 10 with Quarto v. 1.3.142:

D:\quartotest\shinylive>ls
_extensions  test.qmd

D:\quartotest\shinylive>ls _extensions\quarto-ext\shinylive
_extension.yml  resources  shinylive.lua

Here test.qmd is the example from the shinylive github page. Here is the result:

D:\quartotest\shinylive>quarto render test.qmd
pandoc
  to: html
  output-file: test.html
  standalone: true
  section-divs: true
  html-math-method: mathjax
  wrap: none
  default-image-extension: png

metadata
  document-css: false
  link-citations: true
  date-format: long
  lang: en
  title: Shinylive in Quarto example

PandocIOError "pipe" shinylive: createProcess: does not exist (No such file or directory)
Error running filter C:/Users/au234616/AppData/Local/Programs/Quarto/share/filters/main.lua:
PandocLuaError "...shinylive\\_extensions\\quarto-ext\\shinylive\\shinylive.lua:33: attempt to index a nil value (upvalue 'codeblockScript')\nstack traceback:\n\t...shinylive\\_extensions\\quarto-ext\\shinylive\\shinylive.lua:69: in function <...shinylive\\_extensions\\quarto-ext\\shinylive\\shinylive.lua:67>\n\t[C]: in ?\n\t[C]: in method 'walk'\n\t...616/AppData/Local/Programs/Quarto/share/filters/main.lua:153: in function 'run_emulated_filter'\n\t...616/AppData/Local/Programs/Quarto/share/filters/main.lua:403: in local 'callback'\n\t[string \"...\"]:1787: in field 'withScriptFile'\n\t...616/AppData/Local/Programs/Quarto/share/filters/main.lua:406: in upvalue 'run_emulated_filter_chain'\n\t...616/AppData/Local/Programs/Quarto/share/filters/main.lua:443: in function <...616/AppData/Local/Programs/Quarto/share/filters/main.lua:424>"
stack traceback:
        ...616/AppData/Local/Programs/Quarto/share/filters/main.lua:153: in function 'run_emulated_filter'
        ...616/AppData/Local/Programs/Quarto/share/filters/main.lua:403: in local 'callback'
        [string "..."]:1787: in field 'withScriptFile'
        ...616/AppData/Local/Programs/Quarto/share/filters/main.lua:406: in upvalue 'run_emulated_filter_chain'
        ...616/AppData/Local/Programs/Quarto/share/filters/main.lua:443: in function <...616/AppData/Local/Programs/Quarto/share/filters/main.lua:424>
ERROR: Error running 'shinylive' command. Perhaps you need to install the 'shinylive' Python package?

If I recall correctly, I just installed Python 3.11.1 from the official Python site. This is what Quarto finds and uses:

[>] Checking Python 3 installation....OK
      Version: 3.11.1
      Path: C:/Users/au234616/AppData/Local/Programs/Python/Python311/python.exe
      Jupyter: 5.2.0
      Kernels: python3

and you can see here that shinylive is installed via pip:

D:\quartotest\shinylive>C:/Users/au234616/AppData/Local/Programs/Python/Python311/python.exe -m pip install shinylive
Requirement already satisfied: shinylive in c:\users\au234616\appdata\local\programs\python\python311\lib\site-packages (0.0.8)
Requirement already satisfied: shiny in c:\users\au234616\appdata\local\programs\python\python311\lib\site-packages (from shinylive) (0.2.9)
Requirement already satisfied: click>=8.0.3 in c:\users\au234616\appdata\local\programs\python\python311\lib\site-packages (from shinylive) (8.1.3)
Requirement already satisfied: appdirs>=1.4.4 in c:\users\au234616\appdata\local\programs\python\python311\lib\site-packages (from shinylive) (1.4.4)
Requirement already satisfied: colorama in c:\users\au234616\appdata\local\programs\python\python311\lib\site-packages (from click>=8.0.3->shinylive) (0.4.6)
Requirement already satisfied: typing-extensions>=4.0.1 in c:\users\au234616\appdata\local\programs\python\python311\lib\site-packages (from shiny->shinylive) (4.4.0)
Requirement already satisfied: uvicorn>=0.16.0 in c:\users\au234616\appdata\local\programs\python\python311\lib\site-packages (from shiny->shinylive) (0.20.0)
Requirement already satisfied: starlette>=0.17.1 in c:\users\au234616\appdata\local\programs\python\python311\lib\site-packages (from shiny->shinylive) (0.23.1)
Requirement already satisfied: contextvars>=2.4 in c:\users\au234616\appdata\local\programs\python\python311\lib\site-packages (from shiny->shinylive) (2.4)
Requirement already satisfied: websockets>=10.0 in c:\users\au234616\appdata\local\programs\python\python311\lib\site-packages (from shiny->shinylive) (10.4)
Requirement already satisfied: python-multipart in c:\users\au234616\appdata\local\programs\python\python311\lib\site-packages (from shiny->shinylive) (0.0.5)
Requirement already satisfied: htmltools>=0.1.0.9001 in c:\users\au234616\appdata\local\programs\python\python311\lib\site-packages (from shiny->shinylive) (0.1.2)
Requirement already satisfied: markdown-it-py>=1.1.0 in c:\users\au234616\appdata\local\programs\python\python311\lib\site-packages (from shiny->shinylive) (2.1.0)
Requirement already satisfied: mdit-py-plugins>=0.3.0 in c:\users\au234616\appdata\local\programs\python\python311\lib\site-packages (from shiny->shinylive) (0.3.3)
Requirement already satisfied: linkify-it-py>=1.0 in c:\users\au234616\appdata\local\programs\python\python311\lib\site-packages (from shiny->shinylive) (2.0.0)
Requirement already satisfied: asgiref>=3.5.2 in c:\users\au234616\appdata\local\programs\python\python311\lib\site-packages (from shiny->shinylive) (3.6.0)
Requirement already satisfied: immutables>=0.9 in c:\users\au234616\appdata\local\programs\python\python311\lib\site-packages (from contextvars>=2.4->shiny->shinylive) (0.19)
Requirement already satisfied: packaging>=20.9 in c:\users\au234616\appdata\local\programs\python\python311\lib\site-packages (from htmltools>=0.1.0.9001->shiny->shinylive) (23.0)
Requirement already satisfied: uc-micro-py in c:\users\au234616\appdata\local\programs\python\python311\lib\site-packages (from linkify-it-py>=1.0->shiny->shinylive) (1.0.1)
Requirement already satisfied: mdurl~=0.1 in c:\users\au234616\appdata\local\programs\python\python311\lib\site-packages (from markdown-it-py>=1.1.0->shiny->shinylive) (0.1.2)
Requirement already satisfied: anyio<5,>=3.4.0 in c:\users\au234616\appdata\local\programs\python\python311\lib\site-packages (from starlette>=0.17.1->shiny->shinylive) (3.6.2)
Requirement already satisfied: h11>=0.8 in c:\users\au234616\appdata\local\programs\python\python311\lib\site-packages (from uvicorn>=0.16.0->shiny->shinylive) (0.14.0)
Requirement already satisfied: six>=1.4.0 in c:\users\au234616\appdata\local\programs\python\python311\lib\site-packages (from python-multipart->shiny->shinylive) (1.16.0)
Requirement already satisfied: idna>=2.8 in c:\users\au234616\appdata\local\programs\python\python311\lib\site-packages (from anyio<5,>=3.4.0->starlette>=0.17.1->shiny->shinylive) (3.4)
Requirement already satisfied: sniffio>=1.1 in c:\users\au234616\appdata\local\programs\python\python311\lib\site-packages (from anyio<5,>=3.4.0->starlette>=0.17.1->shiny->shinylive) (1.3.0)

although apparently not picked up correctly:

PS C:\WINDOWS\system32> (gcm shinylive).Source
gcm : The term 'shinylive' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a path was included, verify t
hat the path is correct and try again.
At line:1 char:2
+ (gcm shinylive).Source
+  ~~~~~~~~~~~~~
    + CategoryInfo          : ObjectNotFound: (shinylive:String) [Get-Command], CommandNotFoundException
    + FullyQualifiedErrorId : CommandNotFoundException,Microsoft.PowerShell.Commands.GetCommandCommand

On WSL (Debian Bullseye) I have the same setup although Quarto v. 1.3.154:

(base) stefan@D42538:~/quartotest$ ls
_extensions  test.qmd

(base) stefan@D42538:~/quartotest$ ls _extensions/quarto-ext/shinylive/
_extension.yml  resources  shinylive.lua

At first I'm getting a different error:

(base) stefan@D42538:~/quartotest$ quarto render test.qmd
pandoc
  to: html
  output-file: test.html
  standalone: true
  section-divs: true
  html-math-method: mathjax
  wrap: none
  default-image-extension: png

metadata
  document-css: false
  link-citations: true
  date-format: long
  lang: en
  title: Shinylive in Quarto example

Error running shinylive (error code 2): <no output>
ERROR: The shinylive extension must be used in a Quarto project directory (with a _quarto.yml file).
Error: Option '--sw-dir' requires an argument.
ERROR: Error running 'shinylive' command. Perhaps you need to install the 'shinylive' Python package?
ERROR: expected argument of type string, got nil
Error running filter /opt/quarto/share/filters/main.lua:
PandocLuaError "/opt/quarto/share/pandoc/datadir/_json.lua:167: attempt to get length of a nil value (local 'str')\nstack traceback:\n\t/opt/quarto/share/pandoc/datadir/_json.lua:380: in function '_json.decode'\n\t...uartotest/_extensions/quarto-ext/shinylive/shinylive.lua:60: in global 'getShinyliveBaseDeps'\n\t...uartotest/_extensions/quarto-ext/shinylive/shinylive.lua:35: in global 'ensureShinyliveSetup'\n\t...uartotest/_extensions/quarto-ext/shinylive/shinylive.lua:69: in function <...uartotest/_extensions/quarto-ext/shinylive/shinylive.lua:67>\n\t[C]: in ?\n\t[C]: in method 'walk'\n\t/opt/quarto/share/filters/main.lua:153: in function 'run_emulated_filter'\n\t/opt/quarto/share/filters/main.lua:405: in local 'callback'\n\t[string \"...\"]:1787: in field 'withScriptFile'\n\t/opt/quarto/share/filters/main.lua:408: in upvalue 'run_emulated_filter_chain'\n\t/opt/quarto/share/filters/main.lua:451: in function </opt/quarto/share/filters/main.lua:432>"
stack traceback:
        /opt/quarto/share/filters/main.lua:153: in function 'run_emulated_filter'
        /opt/quarto/share/filters/main.lua:405: in local 'callback'
        [string "..."]:1787: in field 'withScriptFile'
        /opt/quarto/share/filters/main.lua:408: in upvalue 'run_emulated_filter_chain'
        /opt/quarto/share/filters/main.lua:451: in function </opt/quarto/share/filters/main.lua:432>

It seems to require a .yml file, so if I add _quarto.yml with just

filters:
  - shinylive

in it, then I'm getting:

(base) stefan@D42538:~/quartotest$ quarto render test.qmd
pandoc
  to: html
  output-file: test.html
  standalone: true
  section-divs: true
  html-math-method: mathjax
  wrap: none
  default-image-extension: png

metadata
  document-css: false
  link-citations: true
  date-format: long
  lang: en
  title: Shinylive in Quarto example

error: Module not found "file:///opt/quarto/vendor/deno.land/std@0.166.0/io/buffer.ts".
    at file:///home/stefan/.cache/shinylive/shinylive-0.0.11/scripts/codeblock-to-json.js:2763:27
Error running filter /opt/quarto/share/filters/main.lua:
PandocLuaError "Error running quarto (error code 1): <no output>\nstack traceback:\n\t...uartotest/_extensions/quarto-ext/shinylive/shinylive.lua:72: in function <...uartotest/_extensions/quarto-ext/shinylive/shinylive.lua:67>\n\t[C]: in ?\n\t[C]: in method 'walk'\n\t/opt/quarto/share/filters/main.lua:153: in function 'run_emulated_filter'\n\t/opt/quarto/share/filters/main.lua:405: in local 'callback'\n\t[string \"...\"]:1787: in field 'withScriptFile'\n\t/opt/quarto/share/filters/main.lua:408: in upvalue 'run_emulated_filter_chain'\n\t/opt/quarto/share/filters/main.lua:451: in function </opt/quarto/share/filters/main.lua:432>"
stack traceback:
        /opt/quarto/share/filters/main.lua:153: in function 'run_emulated_filter'
        /opt/quarto/share/filters/main.lua:405: in local 'callback'
        [string "..."]:1787: in field 'withScriptFile'
        /opt/quarto/share/filters/main.lua:408: in upvalue 'run_emulated_filter_chain'
        /opt/quarto/share/filters/main.lua:451: in function </opt/quarto/share/filters/main.lua:432>

My Python installation here is via anaconda:

[✓] Checking Python 3 installation....OK
      Version: 3.9.13 (Conda)
      Path: /home/stefan/anaconda3/bin/python
      Jupyter: 4.11.1
      Kernels: python3

but the shinylive package should be installed here:

(base) stefan@D42538:~/quartotest$ which shinylive
/home/stefan/anaconda3/bin/shinylive

For good measure, here's the content of test.qmd:

---
title: Shinylive in Quarto example
format: html
filters:
  - shinylive
---

This is a Shinylive application embedded in a Quarto doc.

```{shinylive-python}
#| standalone: true

from shiny import *

app_ui = ui.page_fluid(
    ui.input_slider("n", "N", 0, 100, 40),
    ui.output_text_verbatim("txt"),
)

def server(input, output, session):
    @output
    @render.text
    def txt():
        return f"The value of n*2 is {input.n() * 2}"

app = App(app_ui, server)

```
cderv commented 1 year ago

It seems to require a .yml file, so if I add _quarto.yml with just

To be sure you test correctly, do create a project and install the extension in the project.

quarto create-project --type default test-shinylive
cd test-shinylive
quarto add quarto-ext/shinylive

Then copy paste content and render the document

quarto render test-shinylive.qmd

Be sure that the python version with shinylive is correctly found (quarto check jupyter will give you the version of Python found)

My Python installation here is via anaconda:

Only on linux or on windows also ? It does not seem so on Windows.

although apparently not picked up correctly:

On Windows, you need to make sure that the binary is found - This means that somehow your anaconda python package Scripts directory must be in PATH. I don't use anaconda myself. You should check what are the folder in your PATH.

I need to upgrade my python debug skill to point you in the right direction on your filesystem by you could also try to look for shinylive.exe on Windows in one of the Python library folder that you know of. The directory where the shinylive.exe should be a directory where other python scripts live and should be in PATH. For example this is where virtualenv.exe lives, or jupyter.exe or even my pip.exe. This is for me in <python installation directory>/Scripts/ - I don't know if anaconda is the same

I believe there could be an issue on windows but I want to check on your environment. I am using pyenv-win to manage my python environment on Windows and there is definitely something not right because of how pyenv-win works.

Not sure you have the same issue. I would like to confirm.

snhansen commented 1 year ago

It's only on WSL that I'm using anaconda, in Windows I have just installed 3.11.1 directly from the official Python site.

On WSL (Debian) I get the same result as before.

(base) stefan@D42538:~/test-shinylive$ quarto render test-shinylive.qmd
pandoc
  to: html
  output-file: test-shinylive.html
  standalone: true
  section-divs: true
  html-math-method: mathjax
  wrap: none
  default-image-extension: png

metadata
  document-css: false
  link-citations: true
  date-format: long
  lang: en
  title: Shinylive in Quarto example

error: Module not found "file:///opt/quarto/vendor/deno.land/std@0.166.0/io/buffer.ts".
    at file:///home/stefan/.cache/shinylive/shinylive-0.0.11/scripts/codeblock-to-json.js:2763:27
Error running filter /opt/quarto/share/filters/main.lua:
PandocLuaError "Error running quarto (error code 1): <no output>\nstack traceback:\n\t...shinylive/_extensions/quarto-ext/shinylive/shinylive.lua:72: in function <...shinylive/_extensions/quarto-ext/shinylive/shinylive.lua:67>\n\t[C]: in ?\n\t[C]: in method 'walk'\n\t/opt/quarto/share/filters/main.lua:153: in function 'run_emulated_filter'\n\t/opt/quarto/share/filters/main.lua:405: in local 'callback'\n\t[string \"...\"]:1787: in field 'withScriptFile'\n\t/opt/quarto/share/filters/main.lua:408: in upvalue 'run_emulated_filter_chain'\n\t/opt/quarto/share/filters/main.lua:451: in function </opt/quarto/share/filters/main.lua:432>"
stack traceback:
        /opt/quarto/share/filters/main.lua:153: in function 'run_emulated_filter'
        /opt/quarto/share/filters/main.lua:405: in local 'callback'
        [string "..."]:1787: in field 'withScriptFile'
        /opt/quarto/share/filters/main.lua:408: in upvalue 'run_emulated_filter_chain'
        /opt/quarto/share/filters/main.lua:451: in function </opt/quarto/share/filters/main.lua:432>

The Python installation that Quarto picks up is this:

(base) stefan@D42538:~/test-shinylive$ quarto check jupyter

[✓] Checking Python 3 installation....OK
      Version: 3.9.13 (Conda)
      Path: /home/stefan/anaconda3/bin/python
      Jupyter: 4.11.1
      Kernels: python3  

and shinylive lives there

(base) stefan@D42538:~/test-shinylive$ which shinylive
/home/stefan/anaconda3/bin/shinylive

On Windows, I added /Scripts/ to PATH and now it's being picked up:

PS C:\WINDOWS\system32> (gcm shinylive).Source
C:\Users\au234616\AppData\Local\Programs\Python\Python311\Scripts\shinylive.exe

I'm still getting an error though:

D:\quartotest\test-shinylive>quarto render test-shinylive.qmd
pandoc
  to: html
  output-file: test-shinylive.html
  standalone: true
  section-divs: true
  html-math-method: mathjax
  wrap: none
  default-image-extension: png

metadata
  document-css: false
  link-citations: true
  date-format: long
  lang: en
  title: Shinylive in Quarto example

Creating directory C:\Users\au234616\AppData\Local\shinylive\shinylive\Cache
C:\Users\au234616\AppData\Local\shinylive\shinylive\Cache\shinylive-0.0.11 does not exist.
Downloading https://github.com/rstudio/shinylive/releases/download/v0.0.11/shinylive-0.0.11.tar.gz...
Unzipping to C:\Users\au234616\AppData\Local\shinylive\shinylive\Cache/
WARNING: Specified QUARTO_DENO does not exist, using built in deno
error: Module not found "file:///C:/Users/au234616/AppData/Local/Programs/Quarto/vendor/deno.land/std@0.166.0/io/buffer.ts".
    at file:///C:/Users/au234616/AppData/Local/shinylive/shinylive/Cache/shinylive-0.0.11/scripts/codeblock-to-json.js:2763:27
Error running filter C:/Users/au234616/AppData/Local/Programs/Quarto/share/filters/main.lua:
PandocLuaError "Error running quarto (error code 1): <no output>\nstack traceback:\n\t...shinylive\\_extensions\\quarto-ext\\shinylive\\shinylive.lua:72: in function <...shinylive\\_extensions\\quarto-ext\\shinylive\\shinylive.lua:67>\n\t[C]: in ?\n\t[C]: in method 'walk'\n\t...616/AppData/Local/Programs/Quarto/share/filters/main.lua:153: in function 'run_emulated_filter'\n\t...616/AppData/Local/Programs/Quarto/share/filters/main.lua:403: in local 'callback'\n\t[string \"...\"]:1787: in field 'withScriptFile'\n\t...616/AppData/Local/Programs/Quarto/share/filters/main.lua:406: in upvalue 'run_emulated_filter_chain'\n\t...616/AppData/Local/Programs/Quarto/share/filters/main.lua:443: in function <...616/AppData/Local/Programs/Quarto/share/filters/main.lua:424>"
stack traceback:
        ...616/AppData/Local/Programs/Quarto/share/filters/main.lua:153: in function 'run_emulated_filter'
        ...616/AppData/Local/Programs/Quarto/share/filters/main.lua:403: in local 'callback'
        [string "..."]:1787: in field 'withScriptFile'
        ...616/AppData/Local/Programs/Quarto/share/filters/main.lua:406: in upvalue 'run_emulated_filter_chain'
        ...616/AppData/Local/Programs/Quarto/share/filters/main.lua:443: in function <...616/AppData/Local/Programs/Quarto/share/filters/main.lua:424>
cderv commented 1 year ago

You're getting both the same error on WSL and WINDOWS now, which is not related to shinylive not being found:

error: Module not found "file:///C:/Users/au234616/AppData/Local/Programs/Quarto/vendor/deno.land/std@0.166.0/io/buffer.ts". at file:///C:/Users/au234616/AppData/Local/shinylive/shinylive/Cache/shinylive-0.0.11/scripts/codeblock-to-json.js:2763:27

error: Module not found "file:///opt/quarto/vendor/deno.land/std@0.166.0/io/buffer.ts". at file:///home/stefan/.cache/shinylive/shinylive-0.0.11/scripts/codeblock-to-json.js:2763:27

Somehow there is missing module in our deno vendor content. The buffer mentioned is from the JS script we execute from shinylive Python module I believe https://github.com/rstudio/shinylive/blob/99ed7f9ec3ca204a39be99cf80bf5ad71b0376fa/src/scripts/codeblock-to-json.ts#LL14

This should be available from standard library bundle with Quarto as mentioned in doc https://quarto.org/docs/projects/scripts.html#deno-scripts

Can you check what you see in those folder

Somehow I don't the see the std@0.166.0 but std@0-93-0 folder. I am no expert in Deno vendoring, but maybe there is something in the bundle creation or installer.

How did you install or updated Quarto ?

In the meantime, let me try again with latest version of Quarto pre-release

snhansen commented 1 year ago

On Windows I'm pretty sure I just downloaded the latest release .exe on the Quarto site and installed it. I can't recall updating it. Here's the folder you asked about (it's /Quarto/bin/vendor/deno-land and not /Quarto/vendor/deno.land though):

C:\Users\au234616>ls C:\Users\au234616\AppData\Local\Programs\Quarto\bin\vendor\deno-land
std@0-91-0  std@0-93-0  x

On WSL I just downloaded the latest .deb on the Quarto site and installed via dpkg. Here's the folder you asked about (it's /quarto/bin/vendor/deno-land and not /quarto/vendor/deno.land though):

stefan@D42538:~$ ls /opt/quarto/bin/vendor/deno-land/
std@0-91-0  std@0-93-0  x

So it appears there's some deno tools missing and perhaps the path Quarto is looking for is the wrong one.

cderv commented 1 year ago

Here's the folder you asked about (it's /Quarto/bin/vendor/deno-land and not /Quarto/vendor/deno.land though):

Same as mine so ok on that.

Here's the folder you asked about (it's /Quarto/bin/vendor/deno-land and not /Quarto/vendor/deno.land though):

Oddly this is not the one looked for in your error log. Do you have maybe another install of quarto ? did you use the dev version of Quarto at some point and forgot to completely remove or switched path completely ? You should check your PATH maybe.

So it appears there's some deno tools missing and perhaps the path Quarto is looking for is the wrong one.

I believe there is something with the vendored dependency. I don't know why your Quarto is looking for std@0.166.0 and not mine. We updated Deno recently in Quarto and 0.166.0 is now the lib that I believe should be included but it is not. Maybe an issue between dev version and installed version that needs to be sorted out on your side. because if the import maps files from dev version are somehow loaded with your other quarto, then indeed the vendored depencencies won't be the same and won't be found.

I don't think this is a shinylive issue anymore. @cscheid you may know more about that new problem.

snhansen commented 1 year ago

I tried limiting my PATH, so now it's:

stefan@D42538:~/test-shinylive$ echo $PATH
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/usr/lib/wsl/lib

There should only be one quarto binary:

stefan@D42538:~/test-shinylive$ whereis quarto
quarto: /usr/local/bin/quarto /opt/quarto/bin/quarto.js /opt/quarto/bin/quarto

stefan@D42538:~/test-shinylive$ ls -la /usr/local/bin/
total 8
drwxrwsr-x  2 root staff 4096 Feb  6 12:43 .
drwxrwsr-x 10 root staff 4096 Aug 14  2018 ..
lrwxrwxrwx  1 root staff   22 Feb  6 12:43 quarto -> /opt/quarto/bin/quarto

It's still looking for std@0.166.0:

stefan@D42538:~/test-shinylive$ quarto render test-shinylive.qmd
pandoc
  to: html
  output-file: test-shinylive.html
  standalone: true
  section-divs: true
  html-math-method: mathjax
  wrap: none
  default-image-extension: png

metadata
  document-css: false
  link-citations: true
  date-format: long
  lang: en
  title: Shinylive in Quarto example

error: Module not found "file:///opt/quarto/vendor/deno.land/std@0.166.0/io/buffer.ts".
    at file:///home/stefan/.cache/shinylive/shinylive-0.0.11/scripts/codeblock-to-json.js:2763:27
Error running filter /opt/quarto/share/filters/main.lua:
PandocLuaError "Error running quarto (error code 1): <no output>\nstack traceback:\n\t...shinylive/_extensions/quarto-ext/shinylive/shinylive.lua:72: in function <...shinylive/_extensions/quarto-ext/shinylive/shinylive.lua:67>\n\t[C]: in ?\n\t[C]: in method 'walk'\n\t/opt/quarto/share/filters/main.lua:153: in function 'run_emulated_filter'\n\t/opt/quarto/share/filters/main.lua:405: in local 'callback'\n\t[string \"...\"]:1787: in field 'withScriptFile'\n\t/opt/quarto/share/filters/main.lua:408: in upvalue 'run_emulated_filter_chain'\n\t/opt/quarto/share/filters/main.lua:451: in function </opt/quarto/share/filters/main.lua:432>"
stack traceback:
        /opt/quarto/share/filters/main.lua:153: in function 'run_emulated_filter'
        /opt/quarto/share/filters/main.lua:405: in local 'callback'
        [string "..."]:1787: in field 'withScriptFile'
        /opt/quarto/share/filters/main.lua:408: in upvalue 'run_emulated_filter_chain'
        /opt/quarto/share/filters/main.lua:451: in function </opt/quarto/share/filters/main.lua:432>
cderv commented 1 year ago

ok thanks.

Can you share again the quarto version used here ? quarto check install

and also this internal commands to check which paths are looked for

quarto --paths

Also as I believe this is a new issue, can you try that. This should reproduce.

then quarto run test.ts

I think this is a possibly a cache issue - you are possibly encountering now the same issue as this one https://github.com/quarto-dev/quarto-cli/issues/3723

@cscheid are you confirming ? Not sure what is the status on this. I think @wch encountered that issue before but I thought it was fixed.

snhansen commented 1 year ago

Sure, here it is:

stefan@D42538:~$ quarto check install

[✓] Checking Quarto installation......OK
      Version: 1.3.154
      Path: /opt/quarto/bin

[✓] Checking basic markdown render....OK
stefan@D42538:~/test-shinylive$ quarto --paths
/opt/quarto/bin
/opt/quarto/share
stefan@D42538:~$ quarto run test.ts
error: Module not found "file:///opt/quarto/vendor/deno.land/std@0.166.0/io/buffer.ts".
    at file:///home/stefan/test.ts:1:27

Come to think of it, I recently upgraded the WSL dist from Stretch to Bullseye. I can't remember if Quarto was installed prior to the dist upgrade but just mentioning it here in case that could be the culprit. I tried to see if I could reproduce the error starting from scratch in a Docker container but I'm unable to do so. Indeed the following Docker container

FROM docker.io/debian:bullseye
RUN apt-get update && apt-get install -y --no-install-recommends \
        curl \
        nano \
        wget \
        python3 \
        python3-pip
ARG QUARTO_VERSION="1.2.335"
RUN curl -o quarto-linux-amd64.deb -L https://github.com/quarto-dev/quarto-cli/releases/download/v${QUARTO_VERSION}/quarto-${QUARTO_VERSION}-linux-amd64.deb
RUN dpkg -i quarto-linux-amd64.deb
RUN quarto check
RUN python3 -m pip install jupyter
RUN python3 -m pip install shinylive --upgrade
RUN quarto check
RUN quarto create-project --type default test-shinylive
WORKDIR test-shinylive
RUN rm test-shinylive.qmd
ADD test-shinylive.qmd .
RUN quarto install extension quarto-ext/shinylive --no-prompt
CMD ["quarto", "preview", "test-shinylive.qmd", "--port", "8080", "--render", "--host", "0.0.0.0", "--no-browser"]```

runs without issues. Seems like my two installations are somehow borked.

cderv commented 1 year ago

Seems like my two installations are somehow borked.

Definitely something about the caching of the deno_std.

@cscheid will tell us what to do. I can probably guide you to remove the cache to solve your issue, but I prefer that he chimes in in case your state can help us debug further. I would think installing a new quarto should somehow invalidate the caching, or else. At least not create an issue. As we can't reproduce, we kind of need you for going further. Hope that is fine with you to wait a little more.

snhansen commented 1 year ago

Sure thing. I'll stand by for now.

cderv commented 1 year ago

[✓] Checking Quarto installation......OK Version: 1.3.154 Path: /opt/quarto/bin

Can you update to latest version of pre-release. We are at v1.3.186 now. possibly the fix I mentioned was not in 1.3.154

snhansen commented 1 year ago

Alright, that took care of the issue on both Windows and Debian. Guess I should've tried this a while ago, lol. What a ride :-)

cderv commented 1 year ago

yeah I really thought that you had the latest version considering the long discussion, I assume it was something already done.

So I guess we can close this now. Thank you !

snhansen commented 1 year ago

Yeah, sorry about that. Thanks for the patience.

cderv commented 1 year ago

No problem. It is really convenient for us to have someone responsive with an issue we can't reproduce directly, and such issue can be tricky. So it was good search and I learnt too in the process. Thanks !