search

quarto-dev / quarto-cli

Open-source scientific and technical publishing system built on Pandoc.
https://quarto.org
Other
3.9k stars 322 forks source link

Preview in Docker container: function not implemented (os error 38) #2771

Closed jimjam-slam closed 1 year ago

jimjam-slam commented 2 years ago

Bug description

Working in the same rocker-based devcontainer I set up for #2716, I'm finding that attempting to preview a file or project gives me:

$ quarto preview
Preparing to preview

ERROR: Function not implemented (os error 38)

I should note that the first preview successfully renders files (provided the container has enough memory, as per #2716!) first - the problem appears to be specifically with the preview step.

(The devcontainer is based on the rocker/tidyverse:4.2.1 image, which uses Ubuntu 20.04 and Quarto 1.1.189. Like #2716, I'm running it from an M1 Mac, so it's emulating the x86_64 image rather than using a native arm64 one.)

Checklist

jimjam-slam commented 2 years ago

I should also note that I can happily open another web server once the docs are rendered (and forward the port using VSCode's automatic port forwarding) to preview. For example:

# the reproducible container i've set up has `output-dir: out`
$ cd out
$ python3 -m http.server

Serving HTTP on 0.0.0.0 port 8000 (http://0.0.0.0:8000/) ...
eitsupi commented 2 years ago

I would suggest checking to see if it only happens when previewing files on bind mounts or if it also happens with files on containers.

I have been using quarto on Docker containers for several months (I am the maintainer of both rocker/tidyverse and that devcontainer definition) and I don't recall encountering such an error.

jimjam-slam commented 2 years ago

@eitsupi Yep, that seems to be it! When I try to preview a file in the workspace folder (so on the bind mount from the host), I get the error. If I copy that file to an internal container folder, like /home/rstudio, the preview runs fine.

eitsupi commented 2 years ago

In short, it is a common problem that occurs when bind-mounting non-Linux file systems. I am not familiar with it as I don't use macOS, but I would recommend looking into the issue on the https://github.com/microsoft/vscode-remote-release repo, as users of VSCode Dev Containers on a mac will surely face it.

The easiest solution I can come up with is to use Docker volumes without using bind mounts, but I am not sure if this is appropriate when sharing devcontainer.json with others.

I don't think there is anything that can be done on the part of quarto-cli.

jimjam-slam commented 2 years ago

Makes sense. My use case for this and #2716 is trying to ensure our open analyses have a ready-to-go, reproducible build env for Codespaces or local dev containers. I may just need to have a little notice on the README for macOS users.

I am surprised that it doesn't occur when running other web servers out of the bind mount (eg. python3 -m http.server), though!

eitsupi commented 2 years ago

I don't know where the quarto error message is coming from, but given the circumstances, I think it is a synchronization issue between file updates on the container and file updates on the filesystem at the bind-mounted destination, so it only happens when changing files at high speed.

cscheid commented 2 years ago

Our current guess is that filesystem watching does not work well in containers, so we should sniff them and downgrade to polling.

jjallaire commented 1 year ago

I have implemented a polling solution for website/book projects here: https://github.com/quarto-dev/quarto-cli/commit/9e43c01b74ff16784f0598115a107010c9666518

This replaces file watcher based polling entirely (and it is efficient b/c it doesn't need to do a full filesystem scan, but rather only checks files actually referred to by the project).

jimjam-slam commented 1 year ago

I should also mention I have also not had any problems as described above since I started using native ARM64 Docker containers for Quarto on my Apple Silicon Mac (rather than emulating the x64 image)! I think there was something in the mix of filesystem watching + emulation that wasn't playing nicely.