General Remark: Major update in preparation

[pyongjoo]

We are currently re-architecting AQP engines (1) to incorporate more features and (2) have comprehensive unit tests. All this work is being performed in our private repository. In the future, this public repository will focus on supporting various interfaces (e.g., JDBC, Spark, etc.) while core rewriting modules will be pulled from the private repository. However, the core techniques Verdict uses for estimating errors (called variational sampling) will remain the same.

The features that will be supported in the future release include:

• [ ] Progressive updates of answers (a.k.a. online aggregation)
• [ ] Missing group estimation (extremely useful for groupby queries)
• [ ] Specifying target errors prior to querying (which guarantees all answers are highly accurate)
• [ ] Intelligent, workload-specific sample designer (when query logs are available)
• [ ] Automatically detecting the changes to base tables.

This major update can slow down the support for our old code.

[biamalveiro]

These new features are fantastic! Is there as estimated date or timeframe when we can expect them in this public repo?

[pyongjoo]

Our rough plan is to finish by the end of February; however, we cannot say for sure since the plans for all submodules have not been figured out. I'll give a more concrete estimate in the future.