search

observablehq / framework

A static site generator for data apps, dashboards, reports, and more. Observable Framework combines JavaScript on the front-end for interactive graphics with any language on the back-end for data analysis.
https://observablehq.com/framework/
ISC License
2.41k stars 111 forks source link

Technique example: data loader, Python to parquet #1422

Closed allisonhorst closed 2 months ago

allisonhorst commented 4 months ago

Deployed: https://observablehq.observablehq.cloud/framework-example-loader-python-to-parquet/

Fil commented 3 months ago

can we link to https://arrow.apache.org/docs/python/generated/pyarrow.parquet.write_table.html explicitly mentioning that there are many options (and recommend compression); and maybe show compression in action?

allisonhorst commented 3 months ago

@Fil I added the compression codec explicitly in the loader (compression="snappy"), and include a sentence pointing to the write_table docs and different compression algorithms. Look okay?

can we link to https://arrow.apache.org/docs/python/generated/pyarrow.parquet.write_table.html explicitly mentioning that there are many options (and recommend compression); and maybe show compression in action?

jaanli commented 3 months ago

Quick question - would dbt work here?