trinodb / trino

Official repository of Trino, the distributed SQL query engine for big data, formerly known as PrestoSQL (https://trino.io)
https://trino.io
Apache License 2.0
10.49k stars 3.02k forks source link

`partitionValues_parsed` field holds incorrect value with logical column name in case of id and name column mapping mode in Delta Lake #24121

Closed ebyhr closed 2 days ago

ebyhr commented 1 week ago

The checkpoint file should use physical column names and correct values.

{
   "path":"log_date=2024-03-03/20241113_112637_00079_v9t2h_d61a63e0-0b10-47fd-bc53-ad7bab80d362",
   "partitionValues":[
      "(""col-d3204827-65fd-4199-b59c-f789f3b72d4e",
      "2024-03-03"")"
   ],
   "size":277.0,
   "modificationTime":1731497197916.0,
   "dataChange":true,
   "stats":"{\"\"numRecords\"\":1,\"\"minValues\"\":{\"\"col-787739d5-5a80-426e-97a6-51baceb6853f\"\":3},\"\"maxValues\"\":{\"\"col-787739d5-5a80-426e-97a6-51baceb6853f\"\":3},\"\"nullCount\"\":{\"\"col-787739d5-5a80-426e-97a6-51baceb6853f\"\":0}}",
   "partitionValues_parsed":{
      "log_date":"None" ← 
   },
   "stats_parsed":{
      "numRecords":1.0,
      "minValues":{
         "col-787739d5-5a80-426e-97a6-51baceb6853f":3.0
      },
      "maxValues":{
         "col-787739d5-5a80-426e-97a6-51baceb6853f":3.0
      },
      "nullCount":{
         "col-787739d5-5a80-426e-97a6-51baceb6853f":0.0,
         "col-d3204827-65fd-4199-b59c-f789f3b72d4e":"None"
      }
   },
   "tags":[

   ]
}