Open Fokko opened 2 months ago
Looks like the field-id is missing in the Avro metadata generated by iceberg-rust. This should conform the spec: https://iceberg.apache.org/spec/#avro
field-id
avro-tools getschema snap-0-1-a0c0a37d-6828-47c6-80e8-64b0a013fe44.avro 24/04/25 14:58:05 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable { "type" : "record", "name" : "manifest_file", "fields" : [ { "name" : "manifest_path", "type" : "string" }, { "name" : "manifest_length", "type" : "long" }, { "name" : "partition_spec_id", "type" : "int" }, { "name" : "content", "type" : "int" }, { "name" : "sequence_number", "type" : "long" }, { "name" : "min_sequence_number", "type" : "long" }, { "name" : "added_snapshot_id", "type" : "long" }, { "name" : "added_data_files_count", "type" : "int" }, { "name" : "existing_data_files_count", "type" : "int" }, { "name" : "deleted_data_files_count", "type" : "int" }, { "name" : "added_rows_count", "type" : "long" }, { "name" : "existing_rows_count", "type" : "long" }, { "name" : "deleted_rows_count", "type" : "long" }, { "name" : "partitions", "type" : [ "null", { "type" : "array", "items" : { "type" : "record", "name" : "r508", "fields" : [ { "name" : "contains_null", "type" : "boolean" }, { "name" : "contains_nan", "type" : [ "null", "boolean" ], "default" : null }, { "name" : "lower_bound", "type" : [ "null", "bytes" ], "default" : null }, { "name" : "upper_bound", "type" : [ "null", "bytes" ], "default" : null } ] } } ], "default" : null }, { "name" : "key_metadata", "type" : [ "null", "bytes" ], "default" : null } ] }
@Fokko, can you please assign this to me? Thanks!
@s-akhtar-baig Certainly 🙌
Looks like the
field-id
is missing in the Avro metadata generated by iceberg-rust. This should conform the spec: https://iceberg.apache.org/spec/#avro