search

facebookincubator / velox

A composable and fully extensible C++ execution engine library for data management systems.
https://velox-lib.io/
Apache License 2.0
3.53k stars 1.16k forks source link

test(parquet): Reader compatibility tests for metadata #11626

Open zuyu opened 2 days ago

zuyu commented 2 days ago

This is the part 1 for #11590, only covers schema and num of tuples.

netlify[bot] commented 2 days ago

Deploy Preview for meta-velox canceled.

Name Link
Latest commit 0ce0378c7aac67d5f0e0000086e11e22fff70e4a
Latest deploy log https://app.netlify.com/sites/meta-velox/deploys/6740c44a0f35f000084a1fde
majetideepak commented 2 days ago

@zuyu Let's download the parquet-testing repo and test against it. We don't want to check-in the files. We need to build a Velox executable that consumes the parquet-testing directory. Let's add a scheduled job that downloads the parquet-testing repo for this. Similar to https://github.com/facebookincubator/velox/pull/10767/files#diff-f4109e225f86cb2508cacc3bd71f0782d0d42dd5caa207350bc8f72abbb18af3

majetideepak commented 2 days ago

We should aim to remove all the files checked in here https://github.com/facebookincubator/velox/tree/main/velox/dwio/parquet/tests/examples