KxSystems / rkdb

R client for kdb+
https://code.kx.com/q/interfaces
Apache License 2.0
41 stars 30 forks source link

datetime => POSIXct conversion improvement #32

Closed lwshang closed 6 years ago

lwshang commented 6 years ago

The datetime type in q and POSIXct in R are both double precision floating number. But their actual definition is different. datetime represents number of days since 2000.01.01 while POSIXct represents number of seconds since 1970.01.01. So rkdb has to do some floating number arithmetic to complete the conversion.

For a datetime variable y, the conversion procedure can be shown as:

secperday   = 24 * 60 * 60
offset_days = 10957
offset_sec  = secperday *  offset_days

Method A:  ( y * secperday ) + offset_sec
Method B:  ( y + offset_days ) * secperday // currently used in rkdb

The difference between Method A and Method B is only the order of multiplication and addition. Currently, rkdb is using Method B. Consider two datetime variables which are very close to each other, their corresponding R conversion can be the same because Method B do the addition first with shift the floating number to a less precise zone. For example,

> t1 <- execute(h, "`datetime$6803.5601388888890142")
> t2 <- execute(h, "`datetime$6803.5601388888917427")
> t2 == T1
[1] TRUE

If Method A is used, t2 should be greater than t1.