heavyai / heavyai-crossfilter

JavaScript library for exploring large multivariate datasets in the browser.
Other
51 stars 20 forks source link

[FE-9355] smarter timestamp casting #97

Closed int3h closed 5 years ago

int3h commented 5 years ago

:speech_balloon: Description

Per discussion from @wamsiv, timestamp casts (specifically for combo chart range bounds) are now redundant; the Core should automatically cast as appropriate. According to Todd, these CASTs are significantly hurting query performance. So I'm removing that CAST from SQL generation.

Note: I also changed the upper range calculation for time ranges. Previously, we took the ms-precision timestamp (so, seconds + three decimal points of ms precision) and appended the string 999999 (for nanosecond precision), and used <= upperBound. Now, I take the ms-precision timestamp, add 1ms, and use < upperBound.

Merge Checklist

:wrench: Issue(s) fixed:

:smoking: Smoke Test

:ship: Merge

tmostak commented 5 years ago

Can we please time this vs master (and any other proposed options) to ensure it's a performance win? Might try in both cpu and gpu mode at some scale, as well as timestamps of various precisions (0,3,6,9)?