Maybe let plugins define custom serve options?

[simonw]

https://twitter.com/psychemedia/status/1481171650934714370

can extensions be passed their own cli args? eg --ext-tiddlywiki-dbname tiddlywiki2.sqlite ?

I've thought something like this might be useful for other plugins in the past, too.

[simonw]

The challenge here is avoiding clashes. What if a plugin adds an option that I later want to use for a new Datasette core feature? Or what if two plugins define the same option?

Maybe the solution is to make them use namespaces defined by the plugin name.

How about this:

datasette t.db --plugin-setting datasette-tiddlywiki db t.db

It's a bit verbose having an option that itself then takes THREE strings: plugin name, setting name, setting value - but it would work.

[psychemedia]

Is it possible to parse things like --ext-{plugin}-{arg} VALUE ?

[simonw]

I try to stick pretty closely to what Click supports, and Click likes you to define options explicitly so that it can display them in the output of --help.

But... that makes me think that actually showing these options in --help is likely a better idea. My --plugin-setting concept would have help that looks something like this:

% datasette serve --help
  ...
  --plugin-setting <TEXT TEXT TEXT>...     Setting for a specified plugin.

That's not great help!

The alternative would be to allow plugins to register their extra options with the command - which would mean the help output could look like this instead:

% datasette serve --help
  ...
  --tiddlywiki-db <TEXT>    Name of database to use for datasette-tiddlywiki

This feels like a good argument to me for plugins to explicitly register their settings. I'm not sure if I should enforce the name-of-plugin- prefix, or if I should allow plugins to define any setting they like. I'm still nervous about plugins over-riding existing or future default options to that command.

[simonw]

Another option: if I make plugin settings a higher level concept in Datasette than they are at the moment, I could allow them to be set either using --options OR using the existing metadata.yml/json mechanism. https://docs.datasette.io/en/stable/plugins.html#plugin-configuration

I want to make changes to that anyway, because I'm increasingly uncomfortable with plugin settings ending up in the "metadata" mechanism.

[simonw]

You can already run datasette --help-settings to see detailed help on available settings. Maybe datasette --help-plugin-settings could do the same thing for plugin settings?

Or I could even have available plugin settings show up as a list at the bottom of the datasette --help-settings output - which currently looks like this:

% datasette --help-settings
Settings:
  default_page_size            Default page size for the table view
                               (default=100)
  max_returned_rows            Maximum rows that can be returned from a table
                               or custom query (default=1000)
  num_sql_threads              Number of threads in the thread pool for
                               executing SQLite queries (default=3)
  sql_time_limit_ms            Time limit for a SQL query in milliseconds
                               (default=1000)
  default_facet_size           Number of values to return for requested facets
                               (default=30)
  facet_time_limit_ms          Time limit for calculating a requested facet
                               (default=200)
  facet_suggest_time_limit_ms  Time limit for calculating a suggested facet
                               (default=50)
  hash_urls                    Include DB file contents hash in URLs, for far-
                               future caching (default=False)
  allow_facet                  Allow users to specify columns to facet using
                               ?_facet= parameter (default=True)
  allow_download               Allow users to download the original SQLite
                               database files (default=True)
  suggest_facets               Calculate and display suggested facets
                               (default=True)
  default_cache_ttl            Default HTTP cache TTL (used in Cache-Control:
                               max-age= header) (default=5)
  default_cache_ttl_hashed     Default HTTP cache TTL for hashed URL pages
                               (default=31536000)
  cache_size_kb                SQLite cache size in KB (0 == use SQLite
                               default) (default=0)
  allow_csv_stream             Allow .csv?_stream=1 to download all rows
                               (ignoring max_returned_rows) (default=True)
  max_csv_mb                   Maximum size allowed for CSV export in MB - set
                               0 to disable this limit (default=100)
  truncate_cells_html          Truncate cells longer than this in HTML table
                               view - set 0 to disable (default=2048)
  force_https_urls             Force URLs in API output to always use https://
                               protocol (default=False)
  template_debug               Allow display of template debug information
                               with ?_context=1 (default=False)
  trace_debug                  Allow display of SQL trace debug information
                               with ?_trace=1 (default=False)
  base_url                     Datasette URLs should use this base path
                               (default=/)

[simonw]

The datasette publish --plugin-secret name setting value option already implements something a bit like this. https://docs.datasette.io/en/stable/plugins.html#secret-configuration-values

It's a bit of a messy hack to compensate for metadata being visible. Maybe I could replace that mechanism with the proposed plugin configuration rethink from this issue.

I still like the debug benefits of making plugin settings public - perhaps add a rule that if a plugin setting has a secret: prefix it gets redacted on a new /-/plugin-settings page.

[simonw]

There's actually already a way to move regular Datasette --setting options to a settings.json file thanks to configuration directory mode: https://docs.datasette.io/en/stable/settings.html

Maybe extending that mechanism to handle plugins would be a neat path forward here.