search

buoyant-data / oxbow

Collection of AWS Lambdas for creating and managing Delta tables
https://www.buoyantdata.com
GNU Affero General Public License v3.0
20 stars 6 forks source link

Optionally report file stats when adding parquet files #14

Open rtyler opened 9 months ago

rtyler commented 9 months ago

Right now there is zero introspection of the parquet files as they are added to the transaction log. it could be interesting to add optional file statistics, but that would require opening the parquet file so should not be the default behavior