search

apache / iceberg-python

Apache PyIceberg
https://py.iceberg.apache.org/
Apache License 2.0
309 stars 114 forks source link

Cast 's', 'ms' and 'ns' PyArrow timestamp to 'us' precision on write #848

Open syun64 opened 1 week ago

syun64 commented 1 week ago

Closes https://github.com/apache/iceberg-python/issues/541 and https://github.com/apache/iceberg-python/issues/840

Question: Are timestamp_ns and timestamptz_ns already supported? If so, should we just limit this PR to casting 's' and 'ms' to 'us' precision, and instead introduce the new timestamp_ns types?

Fokko commented 1 week ago

Thanks @syun64 for working on this! 🙌

Question: Are timestamp_ns and timestamptz_ns already supported? If so, should we just limit this PR to casting 's' and 'ms' to 'us' precision, and instead introduce the new timestamp_ns types?

Nanoseconds timestamp is supported in V3. In order to write nanoseconds without downcasting, we need to check if it is a V3 table.