Allow writing dataframes that are either a subset of table schema or in arbitrary order - Githubissues

apache / iceberg-python

Apache PyIceberg

https://py.iceberg.apache.org/

Apache License 2.0

309 stars 114 forks source link

Allow writing dataframes that are either a subset of table schema or in arbitrary order #829

Open kevinjqliu opened 2 weeks ago

kevinjqliu commented 2 weeks ago

Fixes #674

This PR does the following

Fix test for writing a subset of the table schema (#704 did not include the test_ prefix)
Add schema projection when writing to a dataframe who's schema is a subset of the table's schema
Add test for writing table schema in arbitrary order
Remove casting pyarrow dataframe schema (Originally introduced in #523, #807 is the proper fix)
Rewrite the _check_schema_compatible function