Add local folder support

[schloerke]

So users can load data from folders!

Maybe it could be done through a yaml comment?

Ex:

#| label: fig-shiny-spline
#| fig-height: 7
#| standalone: true
#| shinylive-data: grid_pred,sim_val
#| shinylive-pkgs: tidymodels

APP_CODE_HERE

[schloerke]

@georgestagg Is this something that webr should do?

I could imagine bundling all the files via base64 encoding and adding them to a shinylive editor, but I don't see a clear path when exporting the app to a folder (even if the files were copied).

[jcheng5]

@schloerke Do both the shinylive editor case and the exporting to a folder case, both use app.json? If so, is it just the same?

[schloerke]

There is some internal overlap, but I'd look at them as separate processes

[georgestagg]

Is this something that webr should do?

I think this is something webR could handle, in that there are ways to make specific URLs or data files available in the Emscripten VFS though webR's JS API.

We should ensure anything we do can be replicated in Python/Pyodide. Is there already a solution in place for accessing local files/directories in a Shinylive for Python app?

[schloerke]

Is there already a solution in place for accessing local files/directories in a Shinylive for Python app?

Not that I'm aware of. r-shinylive is a full port of py-shinylive.

From what I remember, most python driven development / testing was within a shinylive editor which can create files within the browser. Where many R users are already have their local apps and want to immediately export them, skipping the editor entirely.

[schloerke]

Currently, there are two use cases: Exported app and quarto app

Exported app

Current shallow dir structure tree -L 1:

.
├── app.json
├── edit
├── index.html
├── shinylive
└── shinylive-sw.js
3 directories, 3 files

Questions:

• If I would like to read a csv from server-side, could I include it in the exported folder?
• If I would like to read a www folder image or JS file, could it be included in ./www next to ./app.json?

So theoretical format:

.
├── app.json
├── edit
├── index.html
├── not_so_secret_local_data.csv
├── shinylive
├── shinylive-sw.js
└── www

... Might be better to put everything inside a folder to avoid collision...

.
├── app.json
├── app_files
│   ├── not_so_secret_local_data.csv
│   └── www
├── edit
├── index.html
├── shinylive
└── shinylive-sw.js

Quarto App

The quarto extension can attach dependencies via quarto.doc.add_html_dependency (here).

I have no problem auto uploading the www folder. Maybe due to the nature of it being public, we should only allow for the www to be added as the www is known to be public. Users could source files from the www themselves. (ex: read.csv("www/not_so_secret_local_data.csv")).

We'd need to get the mapping of what is here / local dir to an isolated app dependencies.. Ex map html dependency app-RANDOM-ID (or chunk name) to . within the R application.

---

@georgestagg ... Can webr mount a url to here's file system within the R process.

Both situations could be solved if we can do this.

[georgestagg]

Can webr mount a url to here's file system within the R process.

Yes, in principle Emscripten's FS.createLazyFile() can be used to do this so that when a certain file in the R process's virtual filesystem is read, the file is fetched from the network from a given URL.

The URL must be CORS permitted. If we are using relative URLs I think it should always work.

WebR does not yet expose FS.createLazyFile() on it's own webR.FS API, but it could do so in the future.

For the moment manually creating the file from R works, by running the following inside the R session:

webr::eval_js("
  Module.FS.createLazyFile(
    '/home/web_user',
    'data.RData',
    'https://raw.githubusercontent.com/topepo/shinylive-in-book-test/main/grid_pred.RData',
    true, false
  )
")

Then again inside R, you can read ~/data.RData to get to the content:

> load("data.RData")
> ls()
[1] "grid_pred"

If we do expose FS.createLazyFile(), we wouldn't need the webr::eval_js() step. We could run webR.FS.createLazyFile() directly in the shinylive TypeScript source.

[schloerke]

Thank you for the workarounds!

[schloerke]

To keep similar things co-located... @georgestagg , is there a similar incantation for py-shinylive? Thank you!

[georgestagg]

In principle, I think something like this could work:

from pyodide import code

code.run_js("""
  pyodide.FS.createLazyFile(
    '/home/pyodide',
    'data.csv',
    'https://media.githubusercontent.com/media/datablist/sample-csv-files/main/files/people/people-100.csv',
    true, false
  )
""")

But, I can't see an easy way to get to the pyodide.FS object from inside the Pyodide worker using code.run_js(). So I think you'd need to tweak the worker at https://github.com/posit-dev/shinylive/blob/main/src/pyodide-worker.ts so that either:

• The pyodide object returned by loadPyodide is made available for use inside the worker (e.g. by setting globalThis.pyodide or similar)
• Or, a new type of message is added to handle requests of this type, with a new branch added to the self.onmessage handler that invokes pyodide.FS.createLazyFile() with arguments provided by the message.

[schloerke]

Proposals

app.json should be updated to contain static directories. shinylive assets's runApp() method should handle calling webr's / pyodide's file mounting.

This should resolve both approaches.

./app.json enhancement:

{
  staticPaths: Array[{"urlPath": RelUrl, "fsRoot": AbsPath, "files": Array[AbsPath]}]
}

All files can be read and can not be overwritten / deleted.

Export app

• Copy www/* folder to destination www/*.
• (Let shinylive assets do the file mounting) ... profit

Shinylive ext

• Needs a new read-only tab of "Static Directories"
• Display the www dir is a static directory. Maybe include how many files? Total file size?
  • If only the www dir is allowed, should we instead display the first level of files / folders? Include file size for files (instead of file count)?
• (Let shinylive assets do the file mounting) ... profit

[schloerke]

Update

For exporting an app, this should already work. The files in the app's folder will automatically be included in app.json.

The difficulty comes in the quarto document. The quarto document has access to the file system, so we should be able to add the folders via a #| shinylive-static: DIR_NAME parameter.

This could be done in https://github.com/posit-dev/shinylive/blob/0cd59dce79a5c980943bba1c8a4af208e462f67b/src/parse-codeblock.ts#L95 and it could add the file results to files in https://github.com/posit-dev/shinylive/blob/0cd59dce79a5c980943bba1c8a4af208e462f67b/src/parse-codeblock.ts .