search

tusharchou / local-data-platform

python library for iceberg lake house on your local
MIT License
8 stars 5 forks source link

0.1.1 Ingest NYC yellow taxi data in local data platform #8

Closed tusharchou closed 1 month ago

tusharchou commented 2 months ago

to-do: Store data in iceberg using pyiceberg and sqlite catalogue

prequel : read parquet source

tusharchou commented 1 month ago

I tried with 0.1.0 version, please fix

tusharchou commented 1 month ago

26

tusharchou commented 1 month ago

https://github.com/tusharchou/local-data-platform/blob/b00fe293acec0c4c5fb116c8f6ae9c2e3df1caf9/local-data-platform/nyc_yellow_taxi.py#L73

poc

tusharchou commented 1 month ago

40 @redpheonixx