search

quarto-dev / quarto-cli

Open-source scientific and technical publishing system built on Pandoc.
https://quarto.org
Other
3.81k stars 309 forks source link

be mindful of case sensitivity in project outputs (was: Website with LICENSE.md and license.qmd fails to render) #6848

Open mine-cetinkaya-rundel opened 1 year ago

mine-cetinkaya-rundel commented 1 year ago

Bug description

I have a website project with a LICENSE.md file and a license.qmd file. The idea was that LICENSE.md is for folks perusing the repo (and used to seeing that file in GitHub repos) and license.qmd can be a page on my actual website. I'm fairly certain this wasn't a problem a couple of months ago when I first put this project together, but today with 1.4.366 it fails to render with the error:

ERROR: NotFound: No such file or directory (os error 2): rename '/Users/mine/Desktop/test-website-license/LICENSE.html' -> '/Users/mine/Desktop/test-website-license/_site/LICENSE.html'

Stack trace:
    at Object.renameSync (ext:deno_fs/30_fs.js:215:7)
    at renderProject (file:///Applications/quarto/bin/quarto.js:76830:22)
    at eventLoopTick (ext:core/01_core.js:181:11)
    at async serveProject (file:///Applications/quarto/bin/quarto.js:95892:24)
    at async Command.fn (file:///Applications/quarto/bin/quarto.js:96481:9)
    at async Command.execute (file:///Applications/quarto/bin/quarto.js:8111:13)
    at async quarto (file:///Applications/quarto/bin/quarto.js:111446:5)
    at async file:///Applications/quarto/bin/quarto.js:111464:9

And this happens whether license.qmd is listed in _quarto.yml or not.

Steps to reproduce

Clone https://github.com/mine-cetinkaya-rundel/test-website-license and render it.

Expected behavior

No response

Actual behavior

No response

Your environment

Quarto check output

Quarto 1.4.366
[✓] Checking versions of quarto binary dependencies...
      Pandoc version 3.1.8: OK
      Dart Sass version 1.55.0: OK
      Deno version 1.33.4: OK
[✓] Checking versions of quarto dependencies......OK
[✓] Checking Quarto installation......OK
      Version: 1.4.366
      Path: /Applications/quarto/bin

[✓] Checking tools....................OK
      TinyTeX: v2023.08
      Chromium: (not installed)

[✓] Checking LaTeX....................OK
      Using: TinyTex
      Path: /Users/mine/Library/TinyTeX/bin/universal-darwin
      Version: 2023

[✓] Checking basic markdown render....OK

[✓] Checking Python 3 installation....OK
      Version: 3.9.6
      Path: /Library/Developer/CommandLineTools/usr/bin/python3
      Jupyter: 5.3.0
      Kernels: python3

[✓] Checking Jupyter engine render....OK

[✓] Checking R installation...........OK
      Version: 4.3.1
      Path: /Library/Frameworks/R.framework/Resources
      LibPaths:
        - /Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/library
      knitr: 1.43
      rmarkdown: 2.24

[✓] Checking Knitr engine render......OK
mcanouil commented 1 year ago

The issue seems to be a conflict, as license.qmd will be rendered as license.md which seems to confuse Quarto since there is LICENSE.md. Renaming license.qmd solves the issue.

This is something that would be handled better via:

Thanks for the report!

cscheid commented 1 year ago

I also wonder if this is a filesystem case sensitive vs case preservation issue, with license.md vs LICENSE.md, etc.

mcanouil commented 1 year ago

FYI, I tried on MacOS same version as Mine, but using VSCode.

cscheid commented 1 year ago

right. macOS is case preserving but not sensitive, so rendering LICENSE.qmd would overwrite license.md. But linux is case sensitive, which would create two separate files LICENSE.md and license.md

(Computers were a mistake)

cscheid commented 10 months ago

Lots of fun little interactions happening here.

I can repro this in 1.4 (and sort of in 1.3, see below), so it's not a regression AFAICT. What's happening here in 1.4 is "simple":

$ quarto render
[1/4] LICENSE.md
[2/4] index.qmd

processing file: index.qmd
1/3
2/3 [unnamed-chunk-1]
3/3
output file: index.knit.md

[3/4] about.qmd

processing file: about.qmd
1/3
2/3 [unnamed-chunk-1]
3/3
output file: about.knit.md

[4/4] license.qmd

The first line of quarto output shows that LICENSE.md is getting rendered by quarto. This is totally a macOS+windows vs Linux thing. LICENSE.md produces LICENSE.html, and license.qmd produces license.html. On Linux, those are different files, but on macOS and windows, they're not.

To avoid emitting LICENSE.html from LICENSE.md, add something like this to _quarto.yml:

project:
  type: website
  render:
    - "*.qmd"
    - "!LICENSE.md"

In 1.3, the story is a bit more fun. In 1.4, we've added a feature that partial outputs are prefixed by their format extension. When license.qmd gets rendered to license.html, in 1.4 we go through license.html.md. In 1.3, we use license.md as the intermediate file, which (on macOS and Windows) overwrites LICENSE.md. The first rendering fails, but the second succeeds (after having damaged your project in the process!):

$ quarto-1.3.450 render
[1/4] LICENSE.md
[2/4] index.qmd

processing file: index.qmd
1/3
2/3 [unnamed-chunk-1]
3/3
output file: index.knit.md

[3/4] about.qmd

processing file: about.qmd
1/3
2/3 [unnamed-chunk-1]
3/3
output file: about.knit.md

[4/4] license.qmd

ERROR: NotFound: No such file or directory (os error 2), rename '/Users/cscheid/Desktop/daily-log/2023/12/04/test-website-license/LICENSE.html' -> '/Users/cscheid/Desktop/daily-log/2023/12/04/test-website-license/_site/LICENSE.html'

NotFound: No such file or directory (os error 2), rename '/Users/cscheid/Desktop/daily-log/2023/12/04/test-website-license/LICENSE.html' -> '/Users/cscheid/Desktop/daily-log/2023/12/04/test-website-license/_site/LICENSE.html'
    at Object.renameSync (deno:runtime/js/30_fs.js:175:9)
    at renderProject (file:///Users/cscheid/repos/github/cscheid/quarto-regress/releases/v1.3.450/bin/quarto.js:86832:22)
    at async Command.fn (file:///Users/cscheid/repos/github/cscheid/quarto-regress/releases/v1.3.450/bin/quarto.js:90856:32)
    at async Command.execute (file:///Users/cscheid/repos/github/cscheid/quarto-regress/releases/v1.3.450/bin/quarto.js:8437:13)
    at async quarto (file:///Users/cscheid/repos/github/cscheid/quarto-regress/releases/v1.3.450/bin/quarto.js:127545:5)
    at async file:///Users/cscheid/repos/github/cscheid/quarto-regress/releases/v1.3.450/bin/quarto.js:127563:9
$ quarto- render
[1/3] index.qmd

processing file: index.qmd
1/3
2/3 [unnamed-chunk-1]
3/3
output file: index.knit.md

[2/3] about.qmd

processing file: about.qmd
1/3
2/3 [unnamed-chunk-1]
3/3
output file: about.knit.md

[3/3] license.qmd

Output created: _site/index.html

That's probably why you thought there was no 1.3 problem.

cscheid commented 10 months ago

What to do?

Two related issues:

  1. In order for us to warn against this kind of problem, we need our project system overhaul.
  2. We need to decide what to do about the OS case sensitivity differences. Options:
    • canonicalize the output capitalization to lowercase
    • accept any capitalization but disallow case-only conflicts
    • have two separate code paths for macOS+Windows and Linux

These are all enhancements, though, and not really a 1.4 bug.