bcgov / rems

An R package to access data from British Columbia's Environmental Monitoring System
Apache License 2.0
19 stars 5 forks source link

ems.sqlite is returning integer64 for date objects #31

Closed boshek closed 5 years ago

boshek commented 5 years ago

I think this is causing problems with ems_posix_numeric:

library(rems)
suppressPackageStartupMessages(library(dplyr))

hist_db <- attach_historic_data()

filtered_historic <- hist_db %>% 
  select(EMS_ID, COLLECTION_START) %>% 
  collect()

class(filtered_historic$COLLECTION_START)
#> [1] "integer64"

foo <- ems_posix_numeric(filtered_historic$COLLECTION_START)
#> Error in as.POSIXct.default(x, origin = "1970/01/01", tz = ems_tz()): do not know how to convert 'x' to class "POSIXct"

bar <- ems_posix_numeric(as.numeric(filtered_historic$COLLECTION_START))

Created on 2019-01-07 by the reprex package (v0.2.1)

Session info ``` r devtools::session_info() #> - Session info ---------------------------------------------------------- #> setting value #> version R version 3.5.2 (2018-12-20) #> os Windows 10 x64 #> system x86_64, mingw32 #> ui RTerm #> language (EN) #> collate English_Canada.1252 #> ctype English_Canada.1252 #> tz America/Los_Angeles #> date 2019-01-07 #> #> - Packages -------------------------------------------------------------- #> package * version date lib source #> assertthat 0.2.0 2017-04-11 [1] CRAN (R 3.5.1) #> backports 1.1.3 2018-12-14 [1] CRAN (R 3.5.1) #> bindr 0.1.1 2018-03-13 [1] CRAN (R 3.5.1) #> bindrcpp 0.2.2 2018-03-29 [1] CRAN (R 3.5.1) #> bit 1.1-14 2018-05-29 [1] CRAN (R 3.5.0) #> bit64 0.9-7 2017-05-08 [1] CRAN (R 3.5.0) #> blob 1.1.1 2018-03-25 [1] CRAN (R 3.5.1) #> callr 3.1.1 2018-12-21 [1] CRAN (R 3.5.2) #> cli 1.0.1 2018-09-25 [1] CRAN (R 3.5.1) #> crayon 1.3.4 2017-09-16 [1] CRAN (R 3.5.1) #> DBI 1.0.0 2018-05-02 [1] CRAN (R 3.5.1) #> dbplyr 1.2.2 2018-07-25 [1] CRAN (R 3.5.1) #> desc 1.2.0 2018-05-01 [1] CRAN (R 3.5.1) #> devtools 2.0.1 2018-10-26 [1] CRAN (R 3.5.2) #> digest 0.6.18 2018-10-10 [1] CRAN (R 3.5.1) #> dplyr * 0.7.8 2018-11-10 [1] CRAN (R 3.5.1) #> evaluate 0.12 2018-10-09 [1] CRAN (R 3.5.1) #> fs 1.2.6 2018-08-23 [1] CRAN (R 3.5.1) #> glue 1.3.0 2018-07-17 [1] CRAN (R 3.5.1) #> highr 0.7 2018-06-09 [1] CRAN (R 3.5.1) #> hms 0.4.2 2018-03-10 [1] CRAN (R 3.5.1) #> htmltools 0.3.6 2017-04-28 [1] CRAN (R 3.5.1) #> httr 1.4.0 2018-12-11 [1] CRAN (R 3.5.1) #> knitr 1.21 2018-12-10 [1] CRAN (R 3.5.1) #> magrittr 1.5 2014-11-22 [1] CRAN (R 3.5.1) #> memoise 1.1.0 2017-04-21 [1] CRAN (R 3.5.1) #> pillar 1.3.1 2018-12-15 [1] CRAN (R 3.5.1) #> pkgbuild 1.0.2 2018-10-16 [1] CRAN (R 3.5.1) #> pkgconfig 2.0.2 2018-08-16 [1] CRAN (R 3.5.1) #> pkgload 1.0.2 2018-10-29 [1] CRAN (R 3.5.1) #> prettyunits 1.0.2 2015-07-13 [1] CRAN (R 3.5.1) #> processx 3.2.1 2018-12-05 [1] CRAN (R 3.5.2) #> ps 1.3.0 2018-12-21 [1] CRAN (R 3.5.2) #> purrr 0.2.5 2018-05-29 [1] CRAN (R 3.5.1) #> R6 2.3.0 2018-10-04 [1] CRAN (R 3.5.1) #> rappdirs 0.3.1 2016-03-28 [1] CRAN (R 3.5.1) #> Rcpp 1.0.0 2018-11-07 [1] CRAN (R 3.5.1) #> readr 1.3.1 2018-12-21 [1] CRAN (R 3.5.2) #> remotes 2.0.2 2018-10-30 [1] CRAN (R 3.5.1) #> rems * 0.4.0.9999 2019-01-07 [1] Github (bcgov/rems@63942fb) #> rlang 0.3.0.1 2018-10-25 [1] CRAN (R 3.5.1) #> rmarkdown 1.11 2018-12-08 [1] CRAN (R 3.5.1) #> rprojroot 1.3-2 2018-01-03 [1] CRAN (R 3.5.1) #> RSQLite 2.1.1 2018-05-06 [1] CRAN (R 3.5.1) #> sessioninfo 1.1.1 2018-11-05 [1] CRAN (R 3.5.1) #> storr 1.2.1 2018-10-18 [1] CRAN (R 3.5.2) #> stringi 1.2.4 2018-07-20 [1] CRAN (R 3.5.1) #> stringr 1.3.1 2018-05-10 [1] CRAN (R 3.5.1) #> testthat 2.0.1 2018-10-13 [1] CRAN (R 3.5.1) #> tibble 2.0.0 2019-01-04 [1] CRAN (R 3.5.2) #> tidyselect 0.2.5 2018-10-11 [1] CRAN (R 3.5.1) #> usethis 1.4.0 2018-08-14 [1] CRAN (R 3.5.1) #> withr 2.1.2 2018-03-15 [1] CRAN (R 3.5.1) #> xfun 0.4 2018-10-23 [1] CRAN (R 3.5.1) #> xml2 1.2.0 2018-01-24 [1] CRAN (R 3.5.1) #> yaml 2.2.0 2018-07-25 [1] CRAN (R 3.5.1) #> #> [1] C:/Users/salbers/R/win-library/3.5 #> [2] C:/Program Files/R/R-3.5.2/library ```

I could write PR by just coercing anything feed to ems_posix_numeric into a number and then add a test. Curious though if this is replicated. I build the database today so that should be current.

ateucher commented 5 years ago

Thanks @boshek - I'm getting the same thing. I didn't think a POSIXct integer would be big enough to be a 64 bit integer.

I guess it would be better to store them as double in the database - I can change that. And probably best to add the as.numeric() to ems_posix_numeric() as a safety catch for those who won't have re-downloaded yet...

Does that make sense to you?