squashql / squashql

Official repository of SquashQL, the SQL query engine for multi-dimensional and hierarchical analysis that empowers your SQL database
https://www.squashql.io
Apache License 2.0
48 stars 7 forks source link

Some calculations could lead to too many prefetch requests #59

Closed paulbares closed 1 year ago

paulbares commented 2 years ago

Related to #16.

Indeed, when trying to compute a comparison

ComparisonMeasureReferencePosition m = QueryBuilder.periodComparison(
        "myMeasure",
        ABSOLUTE_DIFFERENCE,
        sales,
        Map.of(
                "quarter_sales", "q",
                "year_sales", "y-1"
        ));
Period.Quarter period = new Period.Quarter("quarter_sales", "year_sales");

PeriodColumnSetDto periodCS = new PeriodColumnSetDto(period);

query = new QueryDto()
        .table(this.storeName)
        .withColumnSet(ColumnSetKey.PERIOD, periodCS)
        .withCondition("year_sales", eq(2023l))
        .withMeasure(m);

Theoretically, the measure m only requires to prefetch this scope:

QueryScope[tableDto=TableDto(name=myAwesomeStore, joins=[]), subQuery=null, columns=[Field[name=year_sales, type=long], Field[name=quarter_sales, type=int]], conditions={}

Measure sales.

However, the way the query plan is build currently leads to:

This is due to the fact that for each primitive measure, we always prefetch their values for the scope of the query (done during the build of the query plan). What could be done instead but it would require some changes and might introduce more bugs is to never prefetch anything automatically and make primitive measures behave like computed measures. In MeasureEvaluator, instead of throwing an IllegalStateException, we might recopy the prefetch values in the final table result but we would need to pay attention to the conditions in the original query and apply them here, in AITM instead of letting the DB evaluating the conditions to give us the shape of the final result.

paulbares commented 1 year ago

Not sure it is needed and would bring any value. Postponed for now.