RandomFractals / chicago-crimes

Exploring Chicago crimes dataset with Jupyter notebooks, DuckDB, Malloy and new Panel/PyScript data and dashboard tools.
GNU Affero General Public License v3.0
38 stars 4 forks source link

Add Feather and Parquet crimes data write and read examples to PyArrow notebook example #18

Closed RandomFractals closed 1 year ago

RandomFractals commented 1 year ago

added in #17.

PyArrow feather file format info: https://arrow.apache.org/docs/python/feather.html

PyArrow .parquet docs: https://arrow.apache.org/docs/python/parquet.html

RandomFractals commented 1 year ago

Updated crimes data pyarrow notebook with feather and parquet write/read timings, and the resulting file sizes on disc without any compression as shown in the FileSize Explorer side panel:

chicago-crimes-feather-and-parquet

RandomFractals commented 1 year ago

new Feather and Parquet data write/read section with pic in docs: https://github.com/RandomFractals/chicago-crimes#with-pyarrow-feather-and-parquet