simonw / datasette-lite

Datasette running in your browser using WebAssembly and Pyodide
https://lite.datasette.io
Apache License 2.0
335 stars 29 forks source link

?install= option for installing extra packages #37

Closed simonw closed 2 years ago

simonw commented 2 years ago

Many Datasette plugins won't work, but a few might. Supporting ?install=datasette-json-html would be a neat way to try them out.

simonw commented 2 years ago

I tried this prototype:

diff --git a/index.html b/index.html
index 5a6eac5..de860bb 100644
--- a/index.html
+++ b/index.html
@@ -97,8 +97,9 @@ const urlParams = new URLSearchParams(location.search);
 const initialUrl = urlParams.get('url');
 const csvUrls = urlParams.getAll('csv');
 const sqlUrls = urlParams.getAll('sql');
+const installUrls = urlParams.getAll('install');

-datasetteWorker.postMessage({type: 'startup', initialUrl, csvUrls, sqlUrls});
+datasetteWorker.postMessage({type: 'startup', initialUrl, csvUrls, sqlUrls, installUrls});

 let loadingLogs = ["Loading..."];

diff --git a/webworker.js b/webworker.js
index c51354c..1556468 100644
--- a/webworker.js
+++ b/webworker.js
@@ -53,6 +53,11 @@ async function startDatasette(settings) {
     # Workaround for Requested 'h11<0.13,>=0.11', but h11==0.13.0 is already installed
     await micropip.install("h11==0.12.0")
     await micropip.install("datasette==0.62a0")
+    # Install any extra ?install= dependencies
+    install_urls = ${JSON.stringify(settings.installUrls)}
+    if install_urls:
+        for install_url in install_urls:
+            await micropip.install(install_url)
     # Execute any ?sql=URL SQL
     sqls = ${JSON.stringify(sqls)}
     if sqls:

Thin hit this URL: http://localhost:8009/?install=datasette-json-html

And got this error:

Traceback (most recent call last):
  File "/lib/python3.10/asyncio/futures.py", line 201, in result
    raise self._exception
  File "/lib/python3.10/asyncio/tasks.py", line 234, in __step
    result = coro.throw(exc)
  File "/lib/python3.10/site-packages/_pyodide/_base.py", line 500, in eval_code_async
    await CodeRunner(
  File "/lib/python3.10/site-packages/_pyodide/_base.py", line 353, in run_async
    await coroutine
  File "<exec>", line 23, in <module>
  File "/lib/python3.10/asyncio/futures.py", line 284, in __await__
    yield self  # This tells Task to wait for completion.
  File "/lib/python3.10/asyncio/tasks.py", line 304, in __wakeup
    future.result()
  File "/lib/python3.10/asyncio/futures.py", line 201, in result
    raise self._exception
  File "/lib/python3.10/asyncio/tasks.py", line 234, in __step
    result = coro.throw(exc)
  File "/lib/python3.10/site-packages/micropip/_micropip.py", line 183, in install
    transaction = await self.gather_requirements(requirements, ctx, keep_going)
  File "/lib/python3.10/site-packages/micropip/_micropip.py", line 173, in gather_requirements
    await gather(*requirement_promises)
  File "/lib/python3.10/asyncio/futures.py", line 284, in __await__
    yield self  # This tells Task to wait for completion.
  File "/lib/python3.10/asyncio/tasks.py", line 304, in __wakeup
    future.result()
  File "/lib/python3.10/asyncio/futures.py", line 201, in result
    raise self._exception
  File "/lib/python3.10/asyncio/tasks.py", line 232, in __step
    result = coro.send(None)
  File "/lib/python3.10/site-packages/micropip/_micropip.py", line 291, in add_requirement
    await self.add_wheel(
  File "/lib/python3.10/site-packages/micropip/_micropip.py", line 316, in add_wheel
    await self.add_requirement(recurs_req, ctx, transaction)
  File "/lib/python3.10/site-packages/micropip/_micropip.py", line 276, in add_requirement
    raise ValueError(
ValueError: Requested 'datasette', but datasette==0.62a0 is already installed
simonw commented 2 years ago

That looks like it might be related to this issue:

Easiest fix though would be for me to ship Datasette 0.62 with all of the changes that have been accumulating in the various alpha versions.

simonw commented 2 years ago

Now that I've released Datasette 0.62 I got this working. I've pushed the feature but it's not documented yet because I want to try it out a bit more.

simonw commented 2 years ago

Here's a demo: https://lite.datasette.io/?install=datasette-json-html#/content?sql=select+json_object%28%27pre%27%2C+%27hello+there%27%29

simonw commented 2 years ago

Plugins that work, with demo links:

Plugins that will probably work but I need a better test database for them:

simonw commented 2 years ago

Lots of plugins don't work yet because they need to be able to load JavaScript and CSS - datasette-cluster-map and datasette-vega for example. See:

simonw commented 2 years ago

Demo of two plugins installed at once: https://lite.datasette.io/?install=datasette-dateutil&install=datasette-copyable#/fixtures.copyable?sql=select%0A++dateutil_parse(%2210+october+2020+3pm%22)%2C%0A++dateutil_parse_fuzzy(%22This+is+due+10+september%22)%2C%0A++dateutil_parse(%221%2F2%2F2020%22)%2C%0A-++dateutil_parse(%222020-03-04%22)%2C%0A++dateutil_parse_dayfirst(%222020-03-04%22)%3B

simonw commented 2 years ago

This needs to be documented in the README.