observablehq / stdlib

The Observable standard library.
https://observablehq.com/@observablehq/standard-library
ISC License
957 stars 83 forks source link

use DuckDB for more data sources #324

Closed mbostock closed 1 year ago

mbostock commented 1 year ago

This…

🦆🦆🦆

mbostock commented 1 year ago

Per the other PR, I want to change this so that for table cells, we still use the friendly table name rather than __table. This way there is less of a discrepancy between how the data source appears for data table cells vs. SQL cells (though there is still necessarily a difference as we will still use the array-of-objects directly for data table cells, but wrap it in a DuckDBClient for SQL cells).

Alternatively, I wonder if we should just always use DuckDBClient for data table cells… it might simplify a lot of things.

mbostock commented 1 year ago

Updated to pass the friendly name for data table cells, too. And also I dropped the table alias t from the generated SQL to make it prettier (especially since this one letter abbreviation isn’t likely to match the actual name of the table).

mbostock commented 1 year ago

Added support for primitive arrays in 1601b28. I also noticed that the DuckDBClient isn’t handling the FLOAT type correctly, so I think my assumption that DESCRIBE would always return the canonical type names (rather than aliases) listed here may not be true: https://duckdb.org/docs/sql/data_types/overview ☹️

mbostock commented 1 year ago

I think we need to add caching here… otherwise we are re-creating the DuckDBClient every time we run a query.

libbey-observable commented 1 year ago

Looked this over again, looks great!