Build and Render fail when project is on SMB share

guslipkin commented 2 years ago

Bug description

I've tried both book and website projects (essentially the same, I know) on both my work and home systems. Both projects build successfully on local storage. Attempting to preview or render the projects on the SMB share fail with different error messages. Rendering individual files succeeds on the SMB share.

Below is the error generated when trying to render a website or book. This error prints in the Background Jobs tab.

ERROR: No valid input files passed to render
Error in get_parentpid() : attempt to apply non-function
Calls: sourceWithProgress ... rstudio_stdout -> rstudio_detect -> detect_new -> get_parentpid
Execution halted

Some more insight is provided by the below preview error message printed to the console:

ERROR: NotFound: The system cannot find the path specified. (os error 3), stat 'UNC\TRUENAS\Files\guslipkin\guslipkin.github.io'
Error in run_serve_daemon("preview", file, NULL, args, render, port, host,  : 
  Error starting quarto

System:

RStudio 2022.07.1 Build 554
R 4.2.1
Quarto 1.1.179 and 1.1.189
Windows 10 accessing a SMB share

Reproducible example: My website

I'm happy to do more troubleshooting as needed.

Checklist

[X] Please include a minimal, fully reproducible example in a single .qmd file? Please provide the whole file rather than the snippet you believe is causing the issue.
[X] Please format your issue so it is easier for us to read the bug report.
[X] Please document the RStudio IDE version you're running (if applicable), by providing the value displayed in the "About RStudio" main menu dialog?
[X] Please document the operating system you're running. If on Linux, please provide the specific distribution.

andreamazzella commented 1 year ago

Similar error here.

My qmd document correctly renders into a docx when I'm running it from my Documents folder, but when I move it to a shared network directory and I click "Render", after a few seconds I get this error message in the Render tab:

ERROR: No valid input files passed to render

System:

RStudio 2022.07.2 Build 576
R 4.2.1
Windows 10 Enterprise

Also please note similar error in this SO post.

cderv commented 1 year ago

@andreamazzella could you follow the indication here to get the stack trace ? https://quarto.org/bug-reports.html#dont-hold-back-tell-us-anything-you-think-might-make-a-difference

You would need to set environment variable QUARTO_PRINT_STACK to TRUE and then run quarto render on your project

On Windows in a POWERSHELL Terminal you can do

$ENV:QUARTO_PRINT_STACK="true"

before doing a quarto render

andreamazzella commented 1 year ago

@cderv Thanks for the quick reply.

Please bear with me, I'm a complete noob with the terminal - I usually render qmds directly from RStudio.

From Windows PowerShell: I managed to set that environmental variable and cd to the project's path, but then when I try and run quarto render <filename>.qmd I get the error message starting with quarto : The term 'quarto' is not recognized as the name of a cmdlet, function, script file, or operable program.. I'm not sure if this is because I don't have quarto installed outside of RStudio (in which case, I can't install it because I'm on a corporate machine with no admin privileges) or if because I shouldn't have cd to the project's path before running quarto render.

I also tried doing this from the Terminal within RStudio (not sure if it's the right thing to do) but when I try to cd to the project path on the shared drive, I get: '<\\shared.drive\path>' CMD does not support UNC paths as current directories.. Could this be the reason why no valid input files are found?

cderv commented 1 year ago

Please bear with me, I'm a complete noob with the terminal - I usually render qmds directly from RStudio.

No problem. Totally fine to ask for help with this and really happy to provide it.

From Windows PowerShell: I managed to set that environmental variable and cd to the project's path, but then when I try and run quarto render .qmd I get the error message starting with quarto : The term 'quarto' is not recognized as the name of a cmdlet, function, script file, or operable program.. I'm not sure if this is because I don't have quarto installed outside of RStudio

Yes to do it , you would need to have Quarto in PATH which you won't have if you only use the RStudio bundled version.

(in which case, I can't install it because I'm on a corporate machine with no admin privileges) or if because I shouldn't have cd to the project's path before running quarto render.

Quarto can be installed without any admin privileges if you use the Zip bundle (from download page https://quarto.org/docs/download/) as it is at the end only an executable. However, your company policy may be against that and block the executable. This would not put quarto.exe to PATH but you can also do it for your user without any privilege.

I also tried doing this from the Terminal within RStudio (not sure if it's the right thing to do) but when I try to cd to the project path on the shared drive, I get: '<\shared.drive\path>' CMD does not support UNC paths as current directories.. Could this be the reason why no valid input files are found?

Anyway, using Quarto from RStudio terminal is fine for this. What you encounter is a limit as your are using CMD terminal (probably the default) and it does not allow to change directory (cd) into a network drive it seems. You can go to RStudio Terminal options and change the default terminal to be Powershell and try again. If powershell allow for network drive...

TBH using Network drive is probably the issue here - I believe it could even be an issue with Pandoc directly and not just Quarto. I want to check that.

andreamazzella commented 1 year ago

Thanks, that was really helpful. I managed to run this from the Windows PowerShell within RStudio, output below.

PS Microsoft.PowerShell.Core\FileSystem::\\shared_drive\path> $ENV:QUARTO_PRINT_STACK="true"
PS Microsoft.PowerShell.Core\FileSystem::\\shared_drive\path> quarto render factsheet.qmd
ERROR: NotFound: The system cannot find the path specified. (os error 3), stat 'UNC\shared_drive\path\factsheet.qmd'

NotFound: The system cannot find the path specified. (os error 3), stat 'UNC\shared_drive\path\factsheet.qmd'
    at Object.opSync (deno:core/01_core.js:172:12)
    at Object.statSync (deno:runtime/js/30_fs.js:238:22)
    at projectContext (file:///C:/Program%20Files/RStudio/bin/quarto/bin/quarto.js:42899:26)
    at async render (file:///C:/Program%20Files/RStudio/bin/quarto/bin/quarto.js:76761:21)
    at async Command.fn (file:///C:/Program%20Files/RStudio/bin/quarto/bin/quarto.js:77138:32)
    at async Command.execute (file:///C:/Program%20Files/RStudio/bin/quarto/bin/quarto.js:2739:13)
    at async quarto (file:///C:/Program%20Files/RStudio/bin/quarto/bin/quarto.js:101332:5)
    at async file:///C:/Program%20Files/RStudio/bin/quarto/bin/quarto.js:101341:9

EDIT: the full path is in the format \\drive22.organisation.com\this\is\the\path\ In the Windows File Explorer the path looks like this: "This PC > Groupname (\drive22.organisation.com\this\is\the`) (Z:) > path", and if I click on it, it becomes "Z:\path"

cderv commented 1 year ago

Thanks that is helpful. It seems it comes from our Quarto code base and not from Pandoc.

What is exactly the full path of your project by the way ? A possible workaround would be to mount the network drive to a letter (like Z://)

@dragonstyle I'll try to have a look but feel free to jump in with this new information.

kompre commented 1 year ago

Hello,

I think I have a similar problem with a project that it is located in a samba share (e.g. "\fileserver\path\to\quarto-project").

The project will render successfully if to html or docx, but will fail to render to pdf. According to the message error, it fails when it tries to run xelatex because UNC path are not supported. When I map the share to a drive (Z:\) then it work correctly. Here the relevant error message I get (redacted):

running xelatex - 1
  This is XeTeX, Version 3.141592653-2.6-0.999994 (TeX Live 2022) (preloaded format=xelatex)
   restricted \write18 enabled.
  entering extended mode
  '\\FILESERVER\path\to\quarto\project'
  CMD.EXE was started with the above path as the current directory.
  UNC paths are not supported.  Defaulting to Windows directory.

Seems like it is an issue with xelatex calling CMD.EXE, which can't handle unc path?

I am running quarto 1.2.269

cderv commented 1 year ago

it fails when it tries to run xelatex

Do you have the issue with pdflatex or lualatex too ?

Seems like it is an issue with xelatex calling CMD.EXE, which can't handle unc path?

TeX Live will have some script like tlmgr.bat and others and they run on CMD.exe - I believe cd in CMD.EXE can't be used with network drive without them being mapped. Workaround is to use pushd I believe. I'll check maybe if we are calling xelatex on quarto side from a BAT script maybe.

kompre commented 1 year ago

xelatex is called by default by quarto, if no other engine are specified in the yaml. Here the help snippet

I've tried both pdflatex and lualatex: they both works well.

cderv commented 1 year ago

I've tried both pdflatex and lualatex: they both works well.

So definitely something with xelatex

but @kompre you said it was the same issue. Does the stack trace looks the same ? (https://github.com/quarto-dev/quarto-cli/issues/2380#issuecomment-1323704703)

This original issue is about something that probably comes from Quarto, but your issue may be something in xelatex, looking at the log. The error does not seem the same even if the cause (UNC) seems to be the same. Probably different issues. And maybe not about Quarto but how xelatex is doing internally. Having a new issue and a reproducible example would help look into this separately. Thank you.

kompre commented 1 year ago

I thought it could be related because of UNC path and CDM.exe being the reason for failing (like in [here])(https://github.com/quarto-dev/quarto-cli/issues/2380#issuecomment-1323577051), but evidently it is not.

daranzolin commented 1 year ago

Just chiming in to report a similar (or the same) issue on Windows. I had a qmd template rendering correctly in my Documents/ folder, but when I moved the entire project to a shared drive and tried to render again I get:

ERROR: NotFound: The system cannot find the path specified. (os error 3), <my_system_filepath>\WorkforceLinkSF Report.qmd'
Error in `processx::run(quarto_bin, args, echo = TRUE)`:
! System command 'quarto.exe' failed
---
Exit status: 1
stdout & stderr: <printed>
---
Type .Last.error to see the more details.

cderv commented 1 year ago

@daranzolin I see you have correctly hidden you system path in <my_system_filepath>\WorkforceLinkSF Report.qmd but can you share how does it start ? Is this a shared drive mounted as a disk on the windows system or access through no letter directly on the network ? The former does not behave like the latter on Windows, and often issues are with the latter. I want to confirm that.

WorkforceLinkSF Report.qmd

Also I don't think it is that, but if there is a space, did you try without ? though I don't know why it would not work in shared drive but work in drive. Protocol issue maybe (SMB, UNC, ...) - I really to reproduce this kind of setup. 🤔

mlpost commented 1 year ago

@cderv I think I might be experiencing a similar issue as @daranzolin. I can render the qmd template locally, but cannot get a quarto project to render when the project is in Google Drive (G:/My Drive/Data-Projects/Quarto_Test/). I receive this output upon rendering within Rstudio:

processing file: Quarto_Test.qmd

output file: Quarto_Test.knit.md

ERROR: NotFound: The system cannot find the file specified. (os error 2), remove 'G:\My Drive\Data-Projects\Quarto_Test\Quarto_Test_files\execute-results'

NotFound: The system cannot find the file specified. (os error 2), remove 'G:\My Drive\Data-Projects\Quarto_Test\Quarto_Test_files\execute-results'
    at Object.removeSync (deno:runtime/js/30_fs.js:157:9)
    at removeIfExists (file:///C:/PROGRA~1/RStudio/RESOUR~1/app/bin/quarto/bin/quarto.js:5681:14)
    at removeFreezeResults (file:///C:/PROGRA~1/RStudio/RESOUR~1/app/bin/quarto/bin/quarto.js:86411:5)
    at renderExecute (file:///C:/PROGRA~1/RStudio/RESOUR~1/app/bin/quarto/bin/quarto.js:86499:9)
    at async renderFiles (file:///C:/PROGRA~1/RStudio/RESOUR~1/app/bin/quarto/bin/quarto.js:86591:51)
    at async renderProject (file:///C:/PROGRA~1/RStudio/RESOUR~1/app/bin/quarto/bin/quarto.js:86797:25)
    at async renderForPreview (file:///C:/PROGRA~1/RStudio/RESOUR~1/app/bin/quarto/bin/quarto.js:102412:26)
    at async render (file:///C:/PROGRA~1/RStudio/RESOUR~1/app/bin/quarto/bin/quarto.js:102307:20)
    at async preview (file:///C:/PROGRA~1/RStudio/RESOUR~1/app/bin/quarto/bin/quarto.js:102313:21)
    at async Command.fn (file:///C:/PROGRA~1/RStudio/RESOUR~1/app/bin/quarto/bin/quarto.js:103764:9)

I'm using: Windows 11 R 4.3.0 RStudio 2023.06.0 Build 421

mcanouil commented 1 year ago

The team is aware of issue when running on cloud/network drive. Currently, it's not recommended to work in such environments if you are able to do otherwise.

quarto-dev / quarto-cli

Build and Render fail when project is on SMB share #2380

Bug description

Checklist