heavyai / heavyai-crossfilter

JavaScript library for exploring large multivariate datasets in the browser.
Other
51 stars 20 forks source link

[FE-8853] Use higher precision when filtering date ranges #87

Closed CaitW closed 5 years ago

CaitW commented 5 years ago

Table chart (and likely other charts) weren't crossfiltering appropriately on high precision timestamps that had greater precision than milliseconds.

For example, when you'd filter on a row in a table chart that involved greater-than-millisecond precision, a duplicate table chart would show zero results. This is because the WHERE clause generated to filter other tables looked like this:

AND ((data_types_basic20.col_ts9_1 >= TIMESTAMP(3) '2019-12-25 23:45:20.977'
        AND data_types_basic20.col_ts9_1 <= TIMESTAMP(3) '2019-12-25 23:45:20.977'))

A condition that would only be met if a result's col_ts9_1 was exactly equal to 2019-12-25 23:45:20.977, or essentially, equal to 2019-12-25 23:45:20.977000000

We need to increase the precision when querying date ranges to TIMESTAMP(9), and query the lower and upper bounds of a millisecond value. Effectively making the above query this:

AND ((data_types_basic20.col_ts9_1 >= TIMESTAMP(9) '2019-12-25 23:45:20.977000000'
        AND data_types_basic20.col_ts9_1 <= TIMESTAMP(9) '2019-12-25 23:45:20.977999999'))