Repeated downloads of the same WASM package

[pawelru]

I have noticed that there are multiple WASM downloads of the same package (of the same version). Is that desirable? Can it be optimised - e.g. cache on the project level?

Just for the record, full log here.

I have multilevel project structure as follows:

book/
├── lvl1/
│   ├── lvl1.1/
│   │   ├── 1-1-1.qmd
│   │   ├── 1-1-2.qmd
│   │   └── ...
│   ├── lvl1.2/
│   │   ├── 1-2-1.qmd
│   │   ├── 1-2-2.qmd
│   │   └── ...
│   └── ...
├── lvl2/
│   ├── lvl2.1/
│   │   ├── 2-1-1.qmd
│   │   ├── 2-1-2.qmd
│   │   └── ...
│   ├── lvl2.2/
│   │   ├── 2-2-1.qmd
│   │   ├── 2-2-2.qmd
│   │   └── ...
│   └── ...
└── ...

Each of the quarto article has a single shinylive-r code chunk.

During the render process, there are multiple .quarto temp dirs created to which there is a separate download process. That means that I am downloading the same package multiple times as well as the whole project consumes a lot of disk space.

After the render it looks more or less like this:

book/
├── lvl1/
│   ├── lvl1.1/
│   │   ├── .quarto/
│   │   │   └── webr/
│   │   │       ├── appdir/
│   │   │       │   └── app.R
│   │   │       └── destdir/
│   │   │           └── shinylive/
│   │   │               └── webr/
│   │   │                   └── packages/
│   │   │                       └── A LOT OF DIRS!!!
│   │   ├── 1-1-1.qmd
│   │   ├── 1-1-2.qmd
│   │   └── ...
│   ├── lvl1.2/
│   │   ├── .quarto/
│   │   │   └── webr/
│   │   │       ├── appdir/
│   │   │       │   └── app.R
│   │   │       └── destdir/
│   │   │           └── shinylive/
│   │   │               └── webr/
│   │   │                   └── packages/
│   │   │                       └── A LOT OF DIRS!!!
│   │   ├── 1-2-1.qmd
│   │   ├── 1-2-2.qmd
│   │   └── ...
│   └── ...
├── lvl2/
│   ├── lvl2.1/
│   │   ├── .quarto/
│   │   │   └── webr/
│   │   │       ├── appdir/
│   │   │       │   └── app.R
│   │   │       └── destdir/
│   │   │           └── shinylive/
│   │   │               └── webr/
│   │   │                   └── packages/
│   │   │                       └── A LOT OF DIRS!!!
│   │   ├── 2-1-1.qmd
│   │   ├── 2-1-2.qmd
│   │   └── ...
│   ├── lvl2.2/
│   │   ├── .quarto/
│   │   │   └── webr/
│   │   │       ├── appdir/
│   │   │       │   └── app.R
│   │   │       └── destdir/
│   │   │           └── shinylive/
│   │   │               └── webr/
│   │   │                   └── packages/
│   │   │                       └── A LOT OF DIRS!!!
│   │   ├── 2-2-1.qmd
│   │   ├── 2-2-2.qmd
│   │   └── ...
│   └── ...
└── ...

Luckily, .quarto dir is gitignored by default so it keeps my git storage safe. My main concern here is the performance.

In my case it was repeated (at least) 14 times:

• downloads in the logs
• results (locally)

  ❯ find . -type f -name "dplyr_1.1.4.data"
  ./book/tables/disposition/.quarto/_webr/destdir/shinylive/webr/packages/dplyr/dplyr_1.1.4.data
  ./book/tables/exposure/.quarto/_webr/destdir/shinylive/webr/packages/dplyr/dplyr_1.1.4.data
  ./book/tables/medical-history/.quarto/_webr/destdir/shinylive/webr/packages/dplyr/dplyr_1.1.4.data
  ./book/tables/ECG/.quarto/_webr/destdir/shinylive/webr/packages/dplyr/dplyr_1.1.4.data
  ./book/tables/risk-management-plan/.quarto/_webr/destdir/shinylive/webr/packages/dplyr/dplyr_1.1.4.data
  ./book/tables/safety/.quarto/_webr/destdir/shinylive/webr/packages/dplyr/dplyr_1.1.4.data
  ./book/tables/adverse-events/.quarto/_webr/destdir/shinylive/webr/packages/dplyr/dplyr_1.1.4.data
  ./book/tables/demography/.quarto/_webr/destdir/shinylive/webr/packages/dplyr/dplyr_1.1.4.data
  ./book/tables/efficacy/.quarto/_webr/destdir/shinylive/webr/packages/dplyr/dplyr_1.1.4.data
  ./book/tables/vital-signs/.quarto/_webr/destdir/shinylive/webr/packages/dplyr/dplyr_1.1.4.data
  ./book/tables/lab-results/.quarto/_webr/destdir/shinylive/webr/packages/dplyr/dplyr_1.1.4.data
  ./book/tables/concomitant-medications/.quarto/_webr/destdir/shinylive/webr/packages/dplyr/dplyr_1.1.4.data
  ./book/_site/site_libs/quarto-contrib/shinylive-0.5.0/shinylive/webr/packages/dplyr/dplyr_1.1.4.data
  ./book/graphs/other/.quarto/_webr/destdir/shinylive/webr/packages/dplyr/dplyr_1.1.4.data
  ./book/graphs/efficacy/.quarto/_webr/destdir/shinylive/webr/packages/dplyr/dplyr_1.1.4.data

• size (locally)

  ❯ find . -type d -name ".quarto" -exec du -sh {} \;
  416M    ./book/tables/disposition/.quarto
  416M    ./book/tables/exposure/.quarto
  416M    ./book/tables/medical-history/.quarto
  416M    ./book/tables/ECG/.quarto
  416M    ./book/tables/risk-management-plan/.quarto
  416M    ./book/tables/safety/.quarto
  416M    ./book/tables/adverse-events/.quarto
  416M    ./book/tables/demography/.quarto
  416M    ./book/tables/efficacy/.quarto
  416M    ./book/tables/vital-signs/.quarto
  416M    ./book/tables/lab-results/.quarto
  416M    ./book/tables/concomitant-medications/.quarto
  424M    ./book/graphs/other/.quarto
  416M    ./book/graphs/efficacy/.quarto
  34M    ./book/.quarto

[georgestagg]

Indeed this is not desirable. I would personally consider this a form of bug.

I think this will need to be handled in https://github.com/posit-dev/r-shinylive/blob/main/R/quarto_ext.R, as it is this file that makes the writes to .quarto/....