Cannot publish document in subfolder

[cbrnr]

Bug description

When publishing a document which is located in a subdirectory, I get the following error with Quarto 1.3 (this worked with Quarto 1.2):

ERROR: NotFound: No such file or directory (os error 2), stat '08/08/08/08_files/libs'

NotFound: No such file or directory (os error 2), stat '08/08/08/08_files/libs'
    at Object.statSync (deno:runtime/js/30_fs.js:322:9)
    at file:///Users/clemens/Applications/quarto/bin/quarto.js:121353:18
    at Array.reduce (<anonymous>)
    at normalizePublishFiles (file:///Users/clemens/Applications/quarto/bin/quarto.js:121351:45)
    at renderForPublish (file:///Users/clemens/Applications/quarto/bin/quarto.js:121303:24)
    at async renderForPublish (file:///Users/clemens/Applications/quarto/bin/quarto.js:109082:24)
    at async handlePublish (file:///Users/clemens/Applications/quarto/bin/quarto.js:108988:26)
    at async publishDocument (file:///Users/clemens/Applications/quarto/bin/quarto.js:121342:38)
    at async publish5 (file:///Users/clemens/Applications/quarto/bin/quarto.js:121438:132)
    at async doPublish (file:///Users/clemens/Applications/quarto/bin/quarto.js:121394:13)

Here, I used

quarto publish 08/08.qmd

However, when running the command from the same folder as the document, everything works:

quarto publish 08.qmd

I've already reported such an issue for an older version (#2623), but it seems like something broke that fix.

OS: macOS Ventura 13.3.1

Output of quarto check:

``` [✓] Checking versions of quarto binary dependencies... Pandoc version 3.1.1: OK Dart Sass version 1.55.0: OK [✓] Checking versions of quarto dependencies......OK [✓] Checking Quarto installation......OK Version: 1.3.340 Path: /Users/clemens/Applications/quarto/bin [✓] Checking basic markdown render....OK [✓] Checking Python 3 installation....OK Version: 3.10.10 Path: /Library/Frameworks/Python.framework/Versions/3.10/bin/python3 Jupyter: (None) Jupyter is not available in this Python installation. Install with python3 -m pip install jupyter [✓] Checking R installation...........OK Version: 4.3.0 Path: /Library/Frameworks/R.framework/Resources LibPaths: - /Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/library knitr: 1.42 rmarkdown: 2.21 [✓] Checking Knitr engine render......OK ```

Checklist

• [ ] Please include a minimal, fully reproducible example in a single .qmd file? Please provide the whole file rather than the snippet you believe is causing the issue.
• [X] Please format your issue so it is easier for us to read the bug report.
• [ ] Please document the RStudio IDE version you're running (if applicable), by providing the value displayed in the "About RStudio" main menu dialog?
• [X] Please document the operating system you're running. If on Linux, please provide the specific distribution.
• [X] Please provide the output of quarto check so we know which version of quarto and its dependencies you're running.

[spather]

@dragonstyle I think your commit may have broken something. I have a repo (https://github.com/spather/sym-conv) that publishes from a sub-folder. My publish task started failing today with this error:

ERROR: NotFound: No such file or directory (os error 2), stat '/home/runner/work/sym-conv/sym-conv/notebooks/sym-conv_files'

The problem is that my project is configured to render to sym-conv/notebooks/_site/ so the path it's looking for should be /home/runner/work/sym-conv/sym-conv/notebooks/_site/sym-conv_files.

Today, quarto latest is 1.3.357 but a few days ago (last time I published successfully) it was 1.3.353. If I pin the quarto version to 1.3.353 my publish task starts succeeding again. So I think something changed between 1.3.353 and 1.3.357 and this was the only commit that seemed related.

Appreciate any help with this.

[dragonstyle]

I will definitely take a look ASAP, thanks for letting us know.

[dragonstyle]

This should be resolved as of 1.3.361 (and our latest pre-release). Thank you very much for reporting the issue so we could fix it quickly!

[spather]

@dragonstyle thanks for the blazing fast fix!

[cbrnr]

I'm sorry to bring this up again 😢, but publishing is (still? again?) not working for me when the document is in a subdirectory (tested with Quarto 1.3.450).

In the following example, I'm in my project root folder, and the Quarto document 06.qmd is in 06/06.qmd. Then I type:

❯ quarto publish 06/06.qmd

Which results in the following error:

ERROR: NotFound: No such file or directory (os error 2), stat '06/06/06_files/libs'

NotFound: No such file or directory (os error 2), stat '06/06/06_files/libs'

Apparently, there's one 06/ too many...

[mcanouil]

Could you elaborate and provide something to reproduce? Here your command does not even include a target to publish which it should.

[cbrnr]

Sure. I didn't include a target, because the problem is not specific to a target (if you don't specify a target, you select it interactively). But here's a reproducible example using quarto-pub.

I have the following directory structure:

~/Desktop ❯ tree quarto-test 
quarto-test
└── 01
    └── 01.qmd

The contents of 01.qmd is not relevant, but you can use:

---
title: "Hello Quarto!"
format: html
author: Some Random Person
published: 2023-11-14
---

## First section
This is just some sample text to demonstrate how it gets rendered using the full HTML page layout.
It's a little wider than the default layout, so more text fits on a page!

To publish the document (and reproduce the error):

1. ❯ cd ~/Desktop/quarto-test
2. ❯ quarto publish quarto-pub 01/01.qmd

This results in the following error:

? Publish with account: › xxxxx.xxxxxx@xxxxx.com
? Document name: › hello-quarto

[✓] Creating quarto-pub document

Rendering for publish:

pandoc 
  to: html
  output-file: 01.html
  standalone: true
  section-divs: true
  html-math-method: mathjax
  wrap: none
  default-image-extension: png

metadata
  document-css: false
  link-citations: true
  date-format: long
  lang: en
  title: Hello Quarto!
  author: Some Random Person
  published: '2023-11-14'

ERROR: NotFound: No such file or directory (os error 2), stat '01/01/01_files/libs'

NotFound: No such file or directory (os error 2), stat '01/01/01_files/libs'
    at Object.statSync (deno:runtime/js/30_fs.js:322:9)
    at file:///Users/clemens/Applications/quarto/bin/quarto.js:121363:18
    at Array.reduce (<anonymous>)
    at normalizePublishFiles (file:///Users/clemens/Applications/quarto/bin/quarto.js:121361:45)
    at renderForPublish (file:///Users/clemens/Applications/quarto/bin/quarto.js:121313:24)
    at async renderForPublish (file:///Users/clemens/Applications/quarto/bin/quarto.js:109092:24)
    at async handlePublish (file:///Users/clemens/Applications/quarto/bin/quarto.js:108998:26)
    at async publishDocument (file:///Users/clemens/Applications/quarto/bin/quarto.js:121352:38)
    at async publish5 (file:///Users/clemens/Applications/quarto/bin/quarto.js:121448:132)
    at async doPublish (file:///Users/clemens/Applications/quarto/bin/quarto.js:121404:13)

There is no problem if I cd into 01 first:

1. ❯ cd ~/Desktop/quarto-test/01
2. ❯ quarto publish quarto-pub 01.qmd

[cbrnr]

@mcanouil can you reproduce the issue, or do you need more information?

[mcanouil]

If you don't specify the root of the project for instance using _quarto.yml Quarto will use the document location as the root directory, thus anything not next to the document won't be found. This is not a bug, i.e., everything works as expected when the "project" is setup as documented in Quarto.

[cbrnr]

Could you be more specific please? I tried creating a _quarto.yml in the root (i.e., quarto-test in my example), but then running quarto publish from that root gives me the following error:

ERROR: YAMLException: expected a single document in the stream, but found more

Stack trace:
    at load$1 (file:///Users/clemens/Applications/quarto/bin/quarto.js:17395:11)
    at buildJsYamlAnnotation (file:///Users/clemens/Applications/quarto/bin/quarto.js:18147:5)
    at readAnnotatedYamlFromMappedString (file:///Users/clemens/Applications/quarto/bin/quarto.js:18067:16)
    at readAndValidateYamlFromMappedString (file:///Users/clemens/Applications/quarto/bin/quarto.js:18405:30)
    at readAndValidateYamlFromFile (file:///Users/clemens/Applications/quarto/bin/quarto.js:20422:50)
    at file:///Users/clemens/Applications/quarto/bin/quarto.js:72289:32
    at projectContext (file:///Users/clemens/Applications/quarto/bin/quarto.js:72130:32)
    at async createPublishOptions (file:///Users/clemens/Applications/quarto/bin/quarto.js:105829:21)
    at async publishAction (file:///Users/clemens/Applications/quarto/bin/quarto.js:105758:28)
    at async Command.fn (file:///Users/clemens/Applications/quarto/bin/quarto.js:105753:9)

I found this documentation page, so any pointers to relevant documentation would also help.

Thanks!

[cbrnr]

And BTW, I want Quarto to use the document location as its reference path. In case it was unclear, everything works if I publish from within the ~/Desktop/quarto-test/01/ directory, but I would like to be able to publish from ~/Desktop/quarto-test/.

[mcanouil]

You need to use quarto publish <service> <document>.

I can't tell you the issue without knowing exactly what's your environment. I used the same thing 1.4 release and it worked fine when adding _quarto.yml.

Also, could you elaborate on the need to execute the command from outside your project directory?

[cbrnr]

You need to use quarto publish <service> <document>.

That's what I do, but for the record, quarto publish <document> also works, because it prompts me for the service.

I can't tell you the issue without knowing exactly what's your environment.

I tried with the latest stable 1.3.450 and the latest pre-release 1.4.507 on both macOS and Linux. It doesn't work.

I used the same thing 1.4 release and it worked fine when adding _quarto.yml.

Would you mind sharing the relevant line(s) that are required to get this to work?

Also, could you elaborate on the need to execute the command from outside your project directory?

I have many chapters for a topic organized in subdirectories. I often want to publish/update many/all of them, and cd-ing into and out of each subdirectory is a lot of work that doesn't seem necessary. For example, from the root I'd like to do:

quarto publish 01/01.qmd
quarto publish 02/02.qmd
...

[mcanouil]

Please share a Git repository mimicking your setup.

My _quarto.yml has nothing int it. For now the only way to reproduce is to not defining the root when the sub directory document has dependencies.

By the way, that the command prompts you to select many things, but here we are trying to solve your issue, thus you need to be explicit about what you are doing and how, otherwise we'll never be able to help you in a "short" time frame.

[cbrnr]

My _quarto.yml has nothing int it.

Ah! Now it works! Where exactly is this behavior described in the docs? I didn't find it unfortunately...

[mcanouil]

Still, it should not matter if it has some values in it as long as those are valid. See https://quarto.org/docs/projects/quarto-projects.html#project-metadata. It is kind of implied in several places: "All Quarto projects include a _quarto.yml configuration file"

[cbrnr]

I completely missed this information, I was totally not aware that Quarto behaves differently in the presence of (even an empty) _quarto.yml. Maybe it would be worth explicitly pointing it out? In any case, thanks a lot for clarifying!

[mcanouil]

FYI, this difference of behaviour for single document and projects is something that the team intends to improve/change over the next year.