search

duckdb / duckdb_iceberg

MIT License
107 stars 18 forks source link

Write Support #37

Open ibotty opened 5 months ago

ibotty commented 5 months ago

Is there write support on the roadmap? If so, is there an ETA? I'll have to plan a future project and would like to keep using duckdb.

samansmink commented 5 months ago

Hi @ibotty! It's something we want to add but we can not give an ETA currently

ibotty commented 5 months ago

Thank you for the answer. I might consider duckdb when time comes.

rustyconover commented 2 months ago

Incrementally, this is getting closer with the COPY function in DuckDB adding support for writing field id metadata for the columns.

There should be an iceberg_schema() function that will return the schema of a table and the appropriate field ids. Then you could write a Parquet file with the right field ids to be added to the Iceberg table.

NielsKorschinsky commented 2 months ago

Hi @rustyconover ,

that are great news! Do I understand you correctly, that this iceberg_schema() function is already available (in a pre-release)? What is the plan with writing the associated parquet file, could this also be done in sync with writing the schema file via duckdb?

rustyconover commented 2 months ago

Hi @NielsKorschinsky,

At this current time the function iceberg_schema() doesn't exist in the extension, but I can see its usefulness to be along the road of writing changes to Iceberg tables.

Rusty