apache / iceberg

Apache Iceberg
https://iceberg.apache.org/
Apache License 2.0
6.49k stars 2.24k forks source link

API: Follow up on adding Variant data type to implement sanitizing for Variant #11479

Open aihuaxu opened 2 weeks ago

aihuaxu commented 2 weeks ago

Feature Request / Improvement

Follow up on https://github.com/apache/iceberg/pull/11324 to implement sanitized version of Variant.

Rather than returning the hash of Variant as a string, it should understand the types within the variant and recursively sanitize the fields. Example:

{
   "id": 3,
   "date": "2024-11-05" 
 } 

=>

{
   hash-id: , (1 digit number)
   hash-date: (date-today) 
 } 

Query engine

None

Willingness to contribute