databrickslabs / tempo

API for manipulating time series on top of Apache Spark: lagged time values, rolling statistics (mean, avg, sum, count, etc), AS OF joins, downsampling, and interpolation
https://pypi.org/project/dbl-tempo
Other
309 stars 53 forks source link

Metadata cleanup #272

Closed tnixon closed 4 months ago

tnixon commented 2 years ago

This is the first big chunk of the v0.2 refactor work. This update includes a major reworking of the metadata structure for TSDF. I wanted to present it internally to the team as a PR to get eyes on this.

THIS IS A WORK IN PROGRESS I have done quite a bit to get test code passing, but not all are working yet. There are still some loose ends to clean up and there will be more changes coming. The focus here should be on the new meta-data structure for TSDF, and its new constructors, and how these will simplify and streamline other code. I invite your reviews, questions and comments - let's get a good discussion going!

Also note - this is not going into master, just into the v0.2-integration branch. This is just a temporary place to integrate lots of changes and get things ready for a final merge to master when all changes are completed.

lgtm-com[bot] commented 2 years ago

This pull request introduces 6 alerts when merging ab1ff0c66396ff981b1a603aadeaee3aa4a7d3b4 into 38ec63ff01afa63fc393db059c3c4684f0cef941 - view on LGTM.com

new alerts:

lgtm-com[bot] commented 2 years ago

This pull request introduces 6 alerts when merging 1459ff52bc6c74099f497c68fbb9cb9832f812c5 into 38ec63ff01afa63fc393db059c3c4684f0cef941 - view on LGTM.com

new alerts:

CLAassistant commented 11 months ago

CLA assistant check
All committers have signed the CLA.