It would be useful for consumers of Iceberg tables to be able to specify additional metadata in data files that enable them to know how to read the files. Some examples of custom metadata include:
Encryption keys required to read the file,
Compression codecs specified on the file without needing to have a specific file extension,
Metadata that's specific to a custom file format. Suppose we supported CSV tables in Iceberg down the road. It would be nice to attach the column delimiter on a per-file basis so that a table can be comprised of multiple files that may not necessarily be uniform in terms of the exact layout, but have compatible schemas.
The custom metadata field should be of type Map<String, String> and can be an optional column.
It would be useful for consumers of Iceberg tables to be able to specify additional metadata in data files that enable them to know how to read the files. Some examples of custom metadata include:
The custom metadata field should be of type Map<String, String> and can be an optional column.