traversc / qs

Quick serialization of R objects
397 stars 19 forks source link

nanotime are not loaded back correctly in qs #23

Closed statquant closed 4 years ago

statquant commented 4 years ago

Hello there seem to be a bug when saving nanotimes in data.tables I am using qs_0.13.1

R> qs::qsave(data.table(x=nanotime(100)), "/tmp/test.qs")
R> qs::qread("/tmp/test.qs")
                               x
1: 1677-09-21T00:11:28.145224192
Warning message:
In `*.integer64`(secs, as.integer64(1e+09)) :
  NAs produced by integer64 overflow

If that helps the same thing works using fst

statquant commented 4 years ago

Also tested with latest from cran and got the same issue

traversc commented 4 years ago

Thank you for bringing it to my attention. It will be fixed in the next version. In the mean time, you can temporarily fix the issue with e.g.:

library(nanotime)
library(data.table)

z <- data.table(x=nanotime(100))
qsave(z, file="/tmp/temp.z")

q <- qread("/tmp/temp.z")
q$x <- asS4(q$x)
print(q)
                                     x
1: 1970-01-01T00:00:00.000000100+00:00
statquant commented 4 years ago

Great Many thanks Just for other people, q$s = asS4(q$x) would trigger a copy of the data.table, the following would work for arbitrary number of nanotime columns

cols = sapply(q, FUN = function(x) class(x) %in% c("nanotime"))
cols = colnames(q)[cols]
if(length(cols) > 0L)
  q[,(cols) := lapply(.SD, FUN = asS4), .SDcols = cols]
traversc commented 4 years ago

Cool, if you don't mind I'll keep it open in case other people have the same issue they can see it.

traversc commented 4 years ago

Should be fixed with ver 0.20.1 on github and hopefully on CRAN soon). :)