search

quarto-dev / quarto-cli

Open-source scientific and technical publishing system built on Pandoc.
https://quarto.org
Other
3.96k stars 326 forks source link

"Start-Process" fails on Windows with julia engine #10111

Open EvoArt opened 4 months ago

EvoArt commented 4 months ago

Bug description

Hi there,

I have recently started trying to use the native julia engine to render quarto docs, but so far have not managed to do so. Currently, the problem seems to be with the Start-Process function.

Any advice would be greatly appreciated, and many, many thanks for the general awesomeness of Quarto.

Steps to reproduce

I have the following test.qmd file 

---
title: "test"
engine: julia
---

```{julia}
2+1
Then I run `quarto render test.qmd`

### Expected behavior

I expect to see the file rendered.

### Actual behavior

Instead I receive following error message:

Starting julia control server process. This might take a while... ERROR: Start-Process : A positional parameter cannot be found that accepts argument 'System.Object[]'.
At line:1 char:1

Stack trace: At line:1 char:1

Your environment

IDE:

VSCode Version: 1.88.1 (user setup) Commit: e170252f762678dec6ca2cc69aba1570769a5d39 Date: 2024-04-10T17:41:02.734Z Electron: 28.2.8 ElectronBuildId: 27744544 Chromium: 120.0.6099.291 Node.js: 18.18.2 V8: 12.0.267.19-electron.0 OS: Windows_NT x64 10.0.19045

OS:

Edition Windows 10 Enterprise Version 22H2 Installed on ‎11/‎03/‎2021 OS build 19045.4529 Experience Windows Feature Experience Pack 1000.19058.1000.0

Hardware:

Device name 5CG0276F8T Processor Intel(R) Core(TM) i7-8665U CPU @ 1.90GHz 2.11 GHz Installed RAM 16.0 GB (15.8 GB usable) Device ID 44dfe63b-6a91-42b9-8fc5-0799aeec7c4d Product ID 00330-80000-00000-AA035 System type 64-bit operating system, x64-based processor

Quarto check output

 C:\testqmd> quarto check
Quarto 1.5.47
[>] Checking versions of quarto binary dependencies...
      Pandoc version 3.2.0: OK
      Dart Sass version 1.70.0: OK
      Deno version 1.41.0: OK
      Typst version 0.11.0: OK
[>] Checking versions of quarto dependencies......OK
[>] Checking Quarto installation......OK
      Version: 1.5.47
      Path: C:\Program Files\Quarto\bin
      CodePage: 1252

[>] Checking tools....................OK
      TinyTeX: (external install)
      Chromium: (not installed)

[>] Checking LaTeX....................OK
      Using: TinyTex
      Path: C:\Users\arn203\AppData\Roaming\TinyTeX\bin\windows\
      Version: 2024

[>] Checking basic markdown render....OK

[>] Checking Python 3 installation....OK
      Version: 3.9.2
      Path: C:/Users/arn203/AppData/Local/Programs/Python/Python39/python.exe
      Jupyter: 4.11.1
      Kernels: julia-1.6, julia-1.7, julia-1.9, julia-12-threads-1.7, julia-8-threads-1.7, julia-image-1.9, julia-omni-1.9, julia_8_threads-1.7, python3

(/) Checking Jupyter engine render....[IPKernelApp] ERROR | No such comm target registered: quarto_kernel_setup
[>] Checking Jupyter engine render....OK

[>] Checking R installation...........OK
      Version: 4.3.1
      Path: C:/PROGRA~1/R/R-43~1.1
      LibPaths:
        - C:/Users/arn203/AppData/Local/R/win-library/4.3
        - C:/Program Files/R/R-4.3.1/library
      knitr: 1.44
      rmarkdown: 2.25

[>] Checking Knitr engine render......OK
EvoArt commented 4 months ago

from this line in the error Start-Process julia -ArgumentList --startup-file=no , --project=C:\Us...

compared with an example from the docs at https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.management/start-process?view=powershell-7.4 

Start-Process -FilePath "`$env:comspec" -ArgumentList "/c dir `"%SystemDrive%\Program Files`""

it looks like it's just that the ArgumentList is not enclosed in quotation marks.

mcanouil commented 4 months ago

What version of PowerShell are you running? Windows native PowerShell is not 7.4, it is 5.1.

$PSVersionTable.PSVersion

Do you have the same error, when running Quarto, for example at the root of a drive? (The issue might be triggered because of something in your file path, but cannot be sure since we don't have it.)

EvoArt commented 4 months ago

Thanks for looking into this. I am on version PowerShell 5.1.

I have been attempting to run this in C:\testqmd\test.qmd since I wondered if there were filepath issues.

EvoArt commented 4 months ago

The same example: -ArgumentList followed by a list enclosed in quotation marks is on the PowerShell 5.1 docs https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.management/start-process?view=powershell-5.1 (example 7).

cderv commented 4 months ago

The Julia Kernel is directly using Powershell and we need to adapt this command

https://github.com/quarto-dev/quarto-cli/blob/87abd1559c957be4d783516148c503164a2f6bcb/src/execute/julia.ts#L298-L330

I believed I tried the Julia notebook and did not get the problem... So maybe there is really a difference between Powershell 7 and 5 🤔

cderv commented 4 months ago

Though PowerShell is the name for the CLI for PowerShell 5 not 7. So Powershell 5 will be always used.

I need to investigate this more to really understand the problem - especially because our Windows tests seems be passing.

devmotion commented 4 months ago

Any update on this issue or any workaround (apart from switching to the Jupyter engine)? I just tried to rerun a very simple Quarto document with the native Julia engine on Windows that was originally written on a Linux computer, but I ran into the same problem.

mcanouil commented 4 months ago

@devmotion Everything / every update are in the thread.

devmotion commented 3 months ago

Thanks, I had hoped there had been already further undocumented progress since the last activity in the thread a month ago.

Based on the error message and the powershell docs, as discussed above it seems that strings in the string array for "-ArgumentList" are incorrectly not enclosed in quotation marks which causes problems e.g. with : in the --project=C:\... argument. I wonder what the value of juliaProject in https://github.com/quarto-dev/quarto-cli/blob/87abd1559c957be4d783516148c503164a2f6bcb/src/execute/julia.ts#L317C26-L317C37 is in the Windows tests on Github - maybe it's a relative path without : which could explain why the problem does not show up in the Github tests?

cderv commented 3 months ago

FWIW I am on Windows, and I don't see this error. Document renders fine for me without error

I'll try to investigate closer - I am not sure why this would be working on my windows and not others if this is a quote issue.

cderv commented 3 months ago

I believe this could be due with Space in the project use, or any other element in the call. I can at least reproduce with a space powershell

> $env:QUARTO_JULIA_PROJECT="C:\Users\chris\Documents\test-dir with space"
> quarto render index.qmd --no-execute-daemon
Starting julia control server process. This might take a while...
ERROR: Start-Process : Impossible de trouver un param�tre positionnel acceptant l'argument ��with��.
Au caract�re Ligne:1 : 1
+ Start-Process julia -ArgumentList --startup-file=no , --project=C:\Us ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidArgument : (:) [Start-Process], ParameterBindingException
    + FullyQualifiedErrorId : PositionalParameterNotFound,Microsoft.PowerShell.Commands.StartProcessCommand

Stack trace:
Au caract�re Ligne:1 : 1
+ Start-Process julia -ArgumentList --startup-file=no , --project=C:\Us ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidArgument : (:) [Start-Process], ParameterBindingException
    + FullyQualifiedErrorId : PositionalParameterNotFound,Microsoft.PowerShell.Commands.StartProcessCommand

    at startOrReuseJuliaServer (file:///C:/Users/chris/Documents/DEV_R/quarto-cli/src/execute/julia.ts:337:15)
    at eventLoopTick (ext:core/01_core.js:153:7)
    at async getJuliaServerConnection (file:///C:/Users/chris/Documents/DEV_R/quarto-cli/src/execute/julia.ts:423:22)
    at async executeJulia (file:///C:/Users/chris/Documents/DEV_R/quarto-cli/src/execute/julia.ts:516:16)
    at async Object.execute (file:///C:/Users/chris/Documents/DEV_R/quarto-cli/src/execute/julia.ts:141:16)
    at async renderExecute (file:///C:/Users/chris/Documents/DEV_R/quarto-cli/src/command/render/render-files.ts:232:25)
    at async renderFileInternal (file:///C:/Users/chris/Documents/DEV_R/quarto-cli/src/command/render/render-files.ts:592:35)
    at async renderFiles (file:///C:/Users/chris/Documents/DEV_R/quarto-cli/src/command/render/render-files.ts:325:9)
    at async renderProject (file:///C:/Users/chris/Documents/DEV_R/quarto-cli/src/command/render/project.ts:441:23)
    at async Command.actionHandler (file:///C:/Users/chris/Documents/DEV_R/quarto-cli/src/command/render/cmd.ts:248:26)

so definitely some quoting issue to fix. I'll target this, and we'll see if it fixes all of your errors here.

devmotion commented 2 months ago

https://github.com/quarto-dev/quarto-cli/issues/10489 seems to be a duplicate. There's a workaround described in https://github.com/quarto-dev/quarto-cli/issues/10489#issuecomment-2288922426 (indeed changing the -ArgumentList) that seems to fix the problem.

pokapokapoka commented 2 months ago

It seems that the resourcePath("julia/quartonotebookrunner.jl") in the environment where the error occurs might include a space in the path. In my environment, Quarto (installed via winget) is located in C:\Program Files\Quarto.

❯ (Get-Command quarto).Source
C:\Program Files\Quarto\bin\quarto.exe

Therefore, the resourcePath("julia/quartonotebookrunner.jl") becomes: C:\Program Files\Quarto\share\julia\quartonotebookrunner.jl which includes a space.