search

criccomini / pymetastore

A Python Client for Hive Metastore
MIT License
9 stars 1 forks source link

Fix TypeStat conversion in metastore.py #27

Closed criccomini closed 1 year ago

criccomini commented 1 year ago

I noticed several of the TypeStats constructed in get_table_stats have incorrect params. Several have cardinality in their constructor, but aren't getting that supplied. Others are being passed numDVs when they don't have that.

criccomini commented 1 year ago

According to this, no cardinality param exists:

https://devdoc.net/bigdata/hive-confluence-20210220/display/Hive/Column+Statistics+in+Hive.html

cpard commented 1 year ago

According to this, no cardinality param exists:

https://devdoc.net/bigdata/hive-confluence-20210220/display/Hive/Column+Statistics+in+Hive.html

yes, HMS is using DV (distinct values) instead of cardinality, I used cardinality as I find it more descriptive than using DV. I'll take a look to see what inconsistencies are there and fix them.

cpard commented 1 year ago

@criccomini check #28

criccomini commented 1 year ago

Closed by https://github.com/recap-build/pymetastore/pull/28