facebookarchive / hive-io-experimental

Hive I/O Library
Other
66 stars 30 forks source link

Publish stats like row count to Metastore #29

Closed groys closed 9 years ago

groys commented 11 years ago

Currently the number of rows in a partition remains empty.

A hive MR job aggregates row count and raw datasize as it writes out the data, publishes this to a temporary mysql table. On the client side the information is aggregated for all mappers/reducers.

Hive-io should do something similar

Relevant hive code is here http://svn.apache.org/repos/asf/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/stats/jdbc/

facebook-github-bot commented 9 years ago

Thank you for reaching out to us and we'd like to first apoligize for not getting back to you sooner. We're closing out older issues since updates to the project may address the issue you opened or they may no longer be relevant. If you're still experiencing the problem, please reopen the issue - thanks!