cube-js / cube

📊 Cube — The Semantic Layer for Building Data Applications
https://cube.dev
Other
17.91k stars 1.77k forks source link

`FILTER_PARAMS` renders date interval boundaries as `NULL` for some data sources #8754

Closed igorlukanin closed 3 weeks ago

igorlukanin commented 4 weeks ago

Describe the bug As reported in Slack, Cube v0.35.81 and above, including Cube v0.36.3, at least for Athena (confirmed), possibly for MS SQL and DuckDB (reported), renders date interval boundaries as NULL in the following case of FILTER_PARAMS usage.

Data model:

cube(`f_clickstream_events`, {
  sql: `SELECT '2024-01-01::TIMESTAMP AS event_ts, 123 AS value'`,

  dimensions: {
    event_ts: {
      sql: `event_ts`,
      type: `time`
    },

    value: {
      sql: `value`,
      type: `number`
    }
  },

  preAggregations: {
    widget: {
      dimensions: [
        value
      ],
      time_dimension: CUBE.event_ts,
      granularity: `day`,
      partition_granularity: `month`,
      build_range_start: {
        sql: `SELECT current_date - interval '180' day`,
      },
      build_range_end: {
        sql: `SELECT current_date + interval '1' day`,
      },
      scheduledRefresh: true,
      refresh_key: {
        every: '4 hours',
        sql: `
          SELECT max(etl_ts) as etl_ts
          FROM dv_marts.f_clickstream_events
          WHERE ${FILTER_PARAMS.f_clickstream_events.event_ts.filter('date(event_ts)')}`,
      },
    },
  },
})

Data source: Athena.

Refresh key query to the data source—see NULLs:

Screenshot 2024-09-27 at 17 36 48

To Reproduce Steps to reproduce the behavior:

  1. Use the data model above
  2. Connect to Athena
  3. Try to build a pre-aggregation
  4. See Cube running the refresh key query shown above

Expected behavior Apparently, Cube rendered boundaries correctly (without NULLs) before v0.35.81.

Version: v0.35.81+, v0.36.3

KSDaemon commented 3 weeks ago

Closed via https://github.com/cube-js/cube/pull/8761