search

tidyverse / lubridate

Make working with dates in R just that little bit easier
https://lubridate.tidyverse.org
GNU General Public License v3.0
727 stars 207 forks source link

Timezones not working since the update to R 4.0.3 #928

Closed rachaelmburke closed 3 years ago

rachaelmburke commented 3 years ago

I updated R yesterday to version 4.0.3 and now my timezone codes are throwing an error in lubridate.

For example a <- "1 Sep 2020 1:00pm" #make a string date

This works fine:

library(lubridate)
dmy_hm(a) # returns date as expected

But this doesn't work:

dmy_hm(a, tz="Africa/Blantyre") #this throws an error

I get the error

error in C_force_tz(time, tz = tzone, roll) : 
  CCTZ: Unrecognized output timezone: "Africa/Blantyre"

For what it's worth, some of the three letter time zone abbreviations seem to work

dmy_hm(a, tz="UTC") #works ok
dmy_hm(a, tz="CET") #works fine
dmy_hm(a, tz="EST") #works fine

But no other time zone specification works (have tried "Europe/London" and "Africa/Harare")

I've checked by system time zone 

Sys.timezone() #returns "Africa/Blantyre" as expected
OlsonNames()#returns 594 timezone names (including "Africa/Blantyre" and everything else I expected)

I'm not sure what's throwing the issue or if it is specific to lubridate, or something else (?)

gaborcsardi commented 3 years ago

I don't see this with CRAN lubridate, on macOS:

❯ a <- "1 Sep 2020 1:00pm" #make a string date
❯ dmy_hm(a, tz="Africa/Blantyre") #this throws an error
[1] "2020-09-01 13:00:00 CAT"

❯ getRversion()
[1] ‘4.0.3’
❯ Sys.timezone()
[1] "Europe/London"

EDIT: this was on Mojave, possibly only an issue on Catalina?

jjallaire commented 3 years ago

I am on Catalina and also unable to repro.

Here was the change in R 4.0.3:

So my guess here is that the error occurs under certain configurations where the MacOS TZ database is corrupted or missing? From the TZDIR docs:

Where R was configured with option --with-internal-tzcode (the default on Windows: recommended on Solaris), the database at file.path(R.home("share"), "zoneinfo") is used by default: file ‘VERSION’ in that directory states the version. That option is also the default on macOS but there whichever is more recent of the system database at ‘/var/db/timezone/zoneinfo’ and that distributed with R is used by default. Environment variable TZDIR can be used to point to a different ‘zoneinfo’ database: value "internal" indicates that the database from the R sources and "macOS" indicates the system database.

@rachaelmburke What is the value of Sys.getenv("TZDIR") on your system? What happens if you do Sys.setenv(TZDIR = "internal") before you run the code? What about with Sys.setenv(TZDIR = "macOS")?

jjallaire commented 3 years ago

The problem appears to occur if the TZDIR environment variable is set to either "internal" or "macOS". Here is the minimum reproducible example:

# Running on Mac OS X Catalina (10.15.5) under R 4.0.3 w/ timezone changes

Sys.getenv("TZDIR")
#> [1] ""
lubridate::dmy_hm("1 Sep 2020 1:00pm", tz="Africa/Blantyre")
#> [1] "2020-09-01 13:00:00 CAT"

# Now in a fresh R session:

Sys.setenv(TZDIR = "internal") # (also fails for "macOS")
lubridate::dmy_hm("1 Sep 2020 1:00pm", tz="Africa/Blantyre")
#> Error in C_force_tz(time, tz = tzone, roll): CCTZ: Unrecognized output timezone: "Africa/Blantyre"
jjallaire commented 3 years ago

Note that cctz requires that the TZDIR by a path not a special value like "internal": https://github.com/google/cctz/blob/742d370708addf54b79ca1b278198d90977d8a29/src/tzfile.h#L25

adamhsparks commented 3 years ago

This bit me as well. I went as far as reinstalling macOS and R on my machine thinking there was something at the OS level that was wrong with my machine since a colleague using Windows 10 couldn't duplicate and it didn't happen on my other Mac. But with fresh installs of Catalina and R 4.0.3 both, this keeps occurring.

gaborcsardi commented 3 years ago

@adamhsparks Did you install R from CRAN or from HomeBrew? Can you try to set TZDIR = "" in your ~/.Renviron file?

adamhsparks commented 3 years ago

R was installed using the Homebrew cask version of R, not the Homebrew formulae.

gsiemon commented 3 years ago

I'm seeing this issue to on R4.0.3. R4.0.2 on the same system doesn't have this issue.

Note that if I run @jjallaire's reprex above I get an error but he doesn't on the same code. Also if I run Sys.getenv("TZDIR") after executing the lubridate statement you can see that the environment variable has been set to "macOS".

> Sys.getenv("TZDIR")
[1] ""
> 
> lubridate::dmy_hm("1 Sep 2020 1:00pm", tz="Africa/Blantyre")
Error in C_force_tz(time, tz = tzone, roll) : 
  CCTZ: Unrecognized output timezone: "Africa/Blantyre"
> 
> Sys.getenv("TZDIR")
[1] "macOS"

Setting TZDIR to internal doesn't seem to help either.

> Sys.getenv("TZDIR")
[1] ""
> Sys.setenv(TZDIR="internal")
> Sys.getenv("TZDIR")
[1] "internal"
> 
> lubridate::dmy_hm("1 Sep 2020 1:00pm", tz="Africa/Blantyre")
Error in C_force_tz(time, tz = tzone, roll) : 
  CCTZ: Unrecognized output timezone: "Africa/Blantyre"
> 
> Sys.getenv("TZDIR")
[1] "internal"

Full disclosure I am testing MacOS 11 on this system. Both 11.0.0 Beta 10 and 11.0.1 Beta 1 have this issue. @jjallaire I note you aren't on the latest Catalina which I think is 10.15.7. Perhaps something has changed in later macOS builds that causes things to break?

R is installed from CRAN.

Adding an explicity TZDIR = "" or TZDIR = "internal" to .Renviron results in the same output.

This does not occur on Windows or Ubuntu 20.04.

adamhsparks commented 3 years ago

I also tried setting TZDIR = "" in my .Renviron. Sys.getenv("TZDIR") still reports "macOS".

gaborcsardi commented 3 years ago

Another idea: can you try setting it as 

TZDIR="/Library/Frameworks/R.framework/Resources/share/zoneinfo/"

in ~/.Renviron?

If it is still set to macOS after a restart, then you can try setting it in ~/.Rprofile:

Sys.setenv("TZDIR" = "/Library/Frameworks/R.framework/Resources/share/zoneinfo/")
gsiemon commented 3 years ago

@gaborcsardi Your suggestion to set TZDIR to "/Library/Frameworks/R.framework/Resources/share/zoneinfo/" in .Renviron seems to work!

> Sys.getenv("TZDIR")
[1] "/Library/Frameworks/R.framework/Resources/share/zoneinfo/"
> 
> lubridate::dmy_hm("1 Sep 2020 1:00pm", tz="Africa/Blantyre")
[1] "2020-09-01 13:00:00 CAT"
> 
> Sys.getenv("TZDIR")
[1] "/Library/Frameworks/R.framework/Resources/share/zoneinfo/"
>
gaborcsardi commented 3 years ago

FWIW, I don't see this issue on Big Sur, fully updated according to the update manager, the R 4.0.3 installer from CRAN, and CRAN lubridate. TZDIR is not set by default here.

adamhsparks commented 3 years ago

Yes, confirming, setting TZDIR="/Library/Frameworks/R.framework/Resources/share/zoneinfo/" in .Renviron works for my own computer.

Now, what's the best method to deal with this error in a package that uses lubridate if it occurs on another user's machine?

gaborcsardi commented 3 years ago

Now, what's the best method to deal with this error in a package that uses lubridate if it occurs on another user's machine?

I think this will be probably fixed in lubridate. Until then a workaround would be to wrap calls to lubridate, and in the wrapper set TZDIR to the fixed path (file.path(R.home(), "share", "zoneinfo")), and then reset TZDIR in on.exit(). This can be tedious, though. Also it might not work if people don't use the CRAN installer, but an R version that was configured without the internal time zone DB.

adamhsparks commented 3 years ago

Thank you, @gaborcsardi! I suspected it would be fixed in lubridate. We're not in a hurry to publish the package. I think that I can just keep an eye out for that fix and require that version and if need be implement what you have suggested here.

domq commented 3 years ago

Not sure whether I am on crack or what, but I reinstalled 4.0.3 on Oct, 21 and afaict everything was working great then.

Now that the winter timezone is upon us (in Europe/Zurich), I am starting having the issue as well! Could the two be linked? (This would also explain why the issue cropped up only three days ago)

Just a thought.

adamhsparks commented 3 years ago

I tried it again in the office today on my Mac there that also has Catalina and R4.0.3. No worries.

We don't do DST here in Qld, so no time change for me (at the beginning of October), but could still be related?

gaborcsardi commented 3 years ago

I would think that it is related macOS updates, and some versions of the macOS timezone DB are buggy. Actually, not really buggy, they are just more recent than the ones on other macOS versions/updates, and R decides to use them and sets TZDIR to a value that breaks lubridate/cctz.

FWIW these are my versions on Mojave, these work fine with R:

❯ ls -l /var/db/timezone/
total 0
lrwxr-xr-x  1 root  wheel   35 May  9 09:08 icutz@ -> /var/db/timezone/tz/2020a.1.0/icutz
drwxr-xr-x  4 root  wheel  128 Oct 29 08:57 tz/
lrwxr-xr-x  1 root  wheel   29 Oct 29 08:57 tz_latest@ -> /var/db/timezone/tz/2020d.1.0
lrwxr-xr-x  1 root  wheel   38 May  9 09:08 zoneinfo@ -> /var/db/timezone/tz/2020a.1.0/zoneinfo

(I don't really understand why the two versions are mixed here.)

gsiemon commented 3 years ago

Interesting...

On my broken setup I see this...

>ls -l /var/db/timezone                                   
total 0
lrwxr-xr-x  1 root  wheel  35  1 Nov 14:31 icutz -> /var/db/timezone/tz/2020d.1.0/icutz
drwxr-xr-x  3 root  wheel  96 30 Oct 09:55 tz
lrwxr-xr-x  1 root  wheel  29 27 Oct 21:23 tz_latest -> /var/db/timezone/tz/2020d.1.0
lrwxr-xr-x  1 root  wheel  38  1 Nov 14:31 zoneinfo -> /var/db/timezone/tz/2020d.1.0/zoneinfo
adamhsparks commented 3 years ago

My (broken machine) matches @gsiemon's broken setup.

❯ ls -l /var/db/timezone/                                                                        
total 0
lrwxr-xr-x 1 root wheel 35 Oct 31 22:02 icutz -> /var/db/timezone/tz/2020d.1.0/icutz
drwxr-xr-x 3 root wheel 96 Oct 31 21:26 tz
lrwxr-xr-x 1 root wheel 29 Oct 31 21:26 tz_latest -> /var/db/timezone/tz/2020d.1.0
lrwxr-xr-x 1 root wheel 38 Oct 31 22:02 zoneinfo -> /var/db/timezone/tz/2020d.1.0/zoneinfo
gaborcsardi commented 3 years ago

Yeah, small sample, but it seems that if zoneinfo links to a newer tz db, that will make R set TZDIR incorrectly.

gsiemon commented 3 years ago

I've just been back to my Timemachine backup and the timezone versions updated from 2020a.1.0 to 2020d.1.0 on October 27. I didn't install any updates then so I'm guessing this was pushed from Apple's end. That seems to reconcile others saying that R4.0.3 was working fine and then it suddenly stopped working.

Files with differences between the two versions:

>diff -rq ./2020a.1.0/ /var/db/timezone/tz/2020d.1.0
Files ./2020a.1.0/icutz/icutz44l.dat and /var/db/timezone/tz/2020d.1.0/icutz/icutz44l.dat differ
Files ./2020a.1.0/versions.plist and /var/db/timezone/tz/2020d.1.0/versions.plist differ
Files ./2020a.1.0/zoneinfo/+VERSION and /var/db/timezone/tz/2020d.1.0/zoneinfo/+VERSION differ
Files ./2020a.1.0/zoneinfo/Africa/Algiers and /var/db/timezone/tz/2020d.1.0/zoneinfo/Africa/Algiers differ
Files ./2020a.1.0/zoneinfo/Africa/Casablanca and /var/db/timezone/tz/2020d.1.0/zoneinfo/Africa/Casablanca differ
Files ./2020a.1.0/zoneinfo/Africa/El_Aaiun and /var/db/timezone/tz/2020d.1.0/zoneinfo/Africa/El_Aaiun differ
Files ./2020a.1.0/zoneinfo/America/Dawson and /var/db/timezone/tz/2020d.1.0/zoneinfo/America/Dawson differ
Files ./2020a.1.0/zoneinfo/America/Whitehorse and /var/db/timezone/tz/2020d.1.0/zoneinfo/America/Whitehorse differ
Files ./2020a.1.0/zoneinfo/Antarctica/Casey and /var/db/timezone/tz/2020d.1.0/zoneinfo/Antarctica/Casey differ
Files ./2020a.1.0/zoneinfo/Antarctica/Macquarie and /var/db/timezone/tz/2020d.1.0/zoneinfo/Antarctica/Macquarie differ
Files ./2020a.1.0/zoneinfo/Asia/Gaza and /var/db/timezone/tz/2020d.1.0/zoneinfo/Asia/Gaza differ
Files ./2020a.1.0/zoneinfo/Asia/Hebron and /var/db/timezone/tz/2020d.1.0/zoneinfo/Asia/Hebron differ
Files ./2020a.1.0/zoneinfo/Canada/Yukon and /var/db/timezone/tz/2020d.1.0/zoneinfo/Canada/Yukon differ
Files ./2020a.1.0/zoneinfo/Europe/Budapest and /var/db/timezone/tz/2020d.1.0/zoneinfo/Europe/Budapest differ
Files ./2020a.1.0/zoneinfo/Europe/Monaco and /var/db/timezone/tz/2020d.1.0/zoneinfo/Europe/Monaco differ
Files ./2020a.1.0/zoneinfo/Europe/Paris and /var/db/timezone/tz/2020d.1.0/zoneinfo/Europe/Paris differ
Files ./2020a.1.0/zoneinfo/Pacific/Fiji and /var/db/timezone/tz/2020d.1.0/zoneinfo/Pacific/Fiji differ
Only in ./2020a.1.0/zoneinfo/US: Pacific-New
Files ./2020a.1.0/zoneinfo/leapseconds and /var/db/timezone/tz/2020d.1.0/zoneinfo/leapseconds differ

Most of these just look like routine timezone updates. I'm happy to send through anything that piques your interest.

Edit: I've just reread your post @gaborcsardi. That makes a lot of sense. Newer TZ data causes switch from internal to macOS TZ data and somehow everything breaks.

rachaelmburke commented 3 years ago

Hi... so glad it's not just me!

Sorry to disrupt a "issue" / patch thread with what's now a support question.... but is there any chance someone could explain to me a work around that I can use in the mean time until this get fixed

I can't quite work out how to set or change anything in R.environ or .Rprofile to change TZDIR.

I'm on Catalina 10.15.4

I'm a doctor turned stats / epidemiology person. Learning more all the time about how my computer actually works, but definitely a newbie here. And I have a lot of code for an ongoing project that now doesn't work so well!

Thanks,

Rachael

adamhsparks commented 3 years ago

usethis::edit_r_profile() will open it for you and let you edit the file. You just need to restart RStudio after saving

Sent from my iPhone

On 2 Nov 2020, at 21:56, Rachael Burke notifications@github.com wrote:

 Hi... so glad it's not just me!

Sorry to disrupt a "issue" / patch thread with what's now a support question.... but is there any chance someone could explain to me a work around that I can use in the mean time until this get fixed

I can't quite work out how to set or change anything in R.environ or .Rprofile to change TZDIR.

I'm a doctor turned stats / epidemiology person. Learning more all the time about how my computer actually works, but definitely a newbie here. And I have a lot of code for an ongoing project that now doesn't work so well!

Thanks,

Rachael

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

rachaelmburke commented 3 years ago

Amazing - thank you @adamhsparks (and everyone else). Fixed for now!

vspinu commented 3 years ago

I don't see this as lubridate specific but if you can think of a simple and reliable way to detect TZDIR on mac then I think it would be ok to include it in this onLoad snippet.

vspinu commented 3 years ago

Would be also good to know where exactly those strange "internal" ans "MacOS" values are set and by whom.

mj-ramirez commented 3 years ago

Hm, on macOS HighSierra: CCTZ: Invalid timezone of the input vector: "" since this morning. MJR-Mac-Pro:~ $ ls -l /var/db/timezone/ total 0 lrwxr-xr-x 1 root wheel 35 Nov 2 08:43 icutz -> /var/db/timezone/tz/2020d.1.0/icutz drwxr-xr-x 3 root wheel 96 Nov 2 08:44 tz lrwxr-xr-x 1 root wheel 29 Oct 29 08:48 tz_latest -> /var/db/timezone/tz/2020d.1.0 lrwxr-xr-x 1 root wheel 38 Nov 2 08:43 zoneinfo -> /var/db/timezone/tz/2020d.1.0/zoneinfo

my "/Library/Frameworks/R.framework/" is empty ..... cannot find anywhere else...

changing lubridate commands to as.Posix Rbase does not make a difference

gaborcsardi commented 3 years ago

@mj-ramirez

my "/Library/Frameworks/R.framework/" is empty ..... cannot find anywhere else...

Maybe you installed R from Homebrew? It seems like homebrew builds R without the internal time zone DB, and if the system one fails, I am not sure what you can do.

Nevertheless, try file.path(R.home(), "share", "zoneinfo") and if that directory exists, then set TZDIR to that.

Otherwise you can maybe switch to the CRAN version of R, you can even install it with Homebrew:

brew cask install r
mj-ramirez commented 3 years ago

BINGO! there I was thinking using homebrew is being clever. Thanks! "/usr/local/Cellar/r/4.0.3/lib/R/share/zoneinfo"

aylapear commented 3 years ago

I updated my MacOS yesterday to Catalina 10.15.7 and my R to the newest version 4.0.3 and hit the same issue with the CCTZ error. Downgrading back to R 4.0.2 from CRAN has solved it for me.

metacoretechs commented 3 years ago

I've been using lubridate in R 4.0.3 without error every day including this morning, until a few hours ago when I updated to macOS Catalina 10.15.7 supplemental update, and now I get this Unrecognized output timezone error.

hadley commented 3 years ago

The change in R appears to have occurred here: https://github.com/wch/r-source/commit/92712b530e5f0f3c19b1e83ed86554bbfb6d9095.

I think that should provide the necessary info to reverse this change for lubridate. I think it should be be safe for lubridate to just reset TZDIR on load, but it does make me feel a little nervous that we're changing an environment variable in a way that might affect other code (OTOH we're changing TZDIR back to a point to a directory so it seem like it should be safe)

vspinu commented 3 years ago

That change doesn't make sense to me. It blindly overwrites whatever TZDIR is set to. The coment just below that change implies that it could also affect tzcode and glibc which respect TZDIR

I have added a workaround but I think the issue should be investigated further and reported.

Would appreciate if people on mac could report back if the fix worked for them.

ianmcook commented 3 years ago

The fix works seamlessly for me in R 4.0.3 on macOS Catalina 10.15.7 supplemental update. Thanks!

LeeMendelowitz commented 3 years ago

Hmmm the fix is not working for me in R 4.0.3 on macOS Catalina 10.15.7 supplemental update.

> Sys.getenv("TZDIR")
[1] ""
> library(lubridate)

Attaching package: ‘lubridate’

The following objects are masked from ‘package:base’:

    date, intersect, setdiff, union

> Sys.getenv("TZDIR")
[1] ""
> lubridate::dmy_hm("1 Sep 2020 1:00pm", tz="Africa/Blantyre")
Error in C_force_tz(time, tz = tzone, roll) :
  CCTZ: Unrecognized output timezone: "Africa/Blantyre"

It seems that my TZDIR never got set by .onLoad because my TZDIR is "" and "/usr/share/zoneinfo" exists: https://github.com/tidyverse/lubridate/blob/4c8428b7b3bd30c8789340381ebd9f455308dc27/R/zzz.R#L27-L39

The workaround that does work for me is setting TZDIR at startup:

>  Sys.setenv("TZDIR" = file.path(R.home(), "share", "zoneinfo"))
>  lubridate::dmy_hm("1 Sep 2020 1:00pm", tz="Africa/Blantyre")
[1] "2020-09-01 13:00:00 CAT"

This also works: Sys.setenv("TZDIR" = "/usr/share/zoneinfo")

> sessionInfo()
R version 4.0.3 (2020-10-10)
Platform: x86_64-apple-darwin19.6.0 (64-bit)
Running under: macOS Catalina 10.15.7

Matrix products: default
BLAS:   /usr/local/Cellar/r/4.0.3/lib/R/lib/libRblas.dylib
LAPACK: /usr/local/Cellar/r/4.0.3/lib/R/lib/libRlapack.dylib

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base

other attached packages:
[1] lubridate_1.7.9.9000

loaded via a namespace (and not attached):
[1] compiler_4.0.3 generics_0.1.0 Rcpp_1.0.5
vspinu commented 3 years ago

@LeeMendelowitz

It seems that my TZDIR never got set by .onLoad because my TZDIR is "" and "/usr/share/zoneinfo" exists:

Yes, that is by design because CCTZ is picking it up automatically.

What is the value of TZDIR immediately after the error?

vspinu commented 3 years ago

@LeeMendelowitz could you please try again with the new master?

njtierney commented 3 years ago

Just wanted to chime in that the new build fixes my error:

lubridate::ymd("2018-08-10", tz = "Australia/Melbourne")
#> [1] "2018-08-10 AEST"

Created on 2020-11-10 by the reprex package (v0.3.0)

Session info devtools::session_info() #> ─ Session info ─────────────────────────────────────────────────────────────── #> setting value #> version R version 4.0.3 (2020-10-10) #> os macOS Mojave 10.14.6 #> system x86_64, darwin17.0 #> ui X11 #> language (EN) #> collate en_AU.UTF-8 #> ctype en_AU.UTF-8 #> tz Australia/Melbourne #> date 2020-11-10 #> #> ─ Packages ─────────────────────────────────────────────────────────────────── #> package * version date lib source #> assertthat 0.2.1 2019-03-21 [1] CRAN (R 4.0.0) #> backports 1.1.10 2020-09-15 [1] CRAN (R 4.0.2) #> callr 3.5.1 2020-10-13 [1] CRAN (R 4.0.2) #> cli 2.1.0 2020-10-12 [1] CRAN (R 4.0.2) #> crayon 1.3.4 2017-09-16 [1] CRAN (R 4.0.0) #> desc 1.2.0 2018-05-01 [1] CRAN (R 4.0.0) #> devtools 2.3.2 2020-09-18 [1] CRAN (R 4.0.2) #> digest 0.6.26 2020-10-17 [1] CRAN (R 4.0.2) #> ellipsis 0.3.1 2020-05-15 [1] CRAN (R 4.0.0) #> evaluate 0.14 2019-05-28 [1] CRAN (R 4.0.0) #> fansi 0.4.1 2020-01-08 [1] CRAN (R 4.0.0) #> fs 1.5.0 2020-07-31 [1] CRAN (R 4.0.2) #> generics 0.1.0 2020-10-31 [1] CRAN (R 4.0.2) #> glue 1.4.2 2020-08-27 [1] CRAN (R 4.0.2) #> highr 0.8 2019-03-20 [1] CRAN (R 4.0.0) #> htmltools 0.5.0 2020-06-16 [1] CRAN (R 4.0.1) #> knitr 1.30 2020-09-22 [1] CRAN (R 4.0.2) #> lubridate 1.7.9.9001 2020-11-10 [1] Github (tidyverse/lubridate@8071a1f) #> magrittr 1.5 2014-11-22 [1] CRAN (R 4.0.2) #> memoise 1.1.0 2017-04-21 [1] CRAN (R 4.0.0) #> pkgbuild 1.1.0 2020-07-13 [1] CRAN (R 4.0.2) #> pkgload 1.1.0 2020-05-29 [1] CRAN (R 4.0.0) #> prettyunits 1.1.1 2020-01-24 [1] CRAN (R 4.0.0) #> processx 3.4.4 2020-09-03 [1] CRAN (R 4.0.2) #> ps 1.4.0 2020-10-07 [1] CRAN (R 4.0.2) #> R6 2.4.1 2019-11-12 [1] CRAN (R 4.0.0) #> Rcpp 1.0.5 2020-07-06 [1] CRAN (R 4.0.0) #> remotes 2.2.0 2020-07-21 [1] CRAN (R 4.0.2) #> rlang 0.4.8 2020-10-08 [1] CRAN (R 4.0.2) #> rmarkdown 2.4.6 2020-11-03 [1] Github (rstudio/rmarkdown@7239cea) #> rprojroot 1.3-2 2018-01-03 [1] CRAN (R 4.0.0) #> sessioninfo 1.1.1 2018-11-05 [1] CRAN (R 4.0.0) #> stringi 1.5.3 2020-09-09 [1] CRAN (R 4.0.2) #> stringr 1.4.0 2019-02-10 [1] CRAN (R 4.0.0) #> testthat 2.3.2 2020-03-02 [1] CRAN (R 4.0.0) #> usethis 1.6.3 2020-09-17 [1] CRAN (R 4.0.2) #> withr 2.3.0 2020-09-22 [1] CRAN (R 4.0.2) #> xfun 0.18 2020-09-29 [1] CRAN (R 4.0.2) #> yaml 2.2.1 2020-02-01 [1] CRAN (R 4.0.0) #> #> [1] /Library/Frameworks/R.framework/Versions/4.0/Resources/library
LeeMendelowitz commented 3 years ago

The updated master (tidyverse/lubridate@8071a1f) still isn't working for me:

> Sys.getenv("TZDIR")
[1] ""
> lubridate::dmy_hm("1 Sep 2020 1:00pm", tz="Africa/Blantyre")
Error in C_force_tz(time, tz = tzone, roll) :
  CCTZ: Unrecognized output timezone: "Africa/Blantyre"
> Sys.getenv("TZDIR")
[1] "macOS"
sessionInfo

``` > devtools::session_info() ─ Session info ─────────────────────────────────────────────────────────────── setting value version R version 4.0.3 (2020-10-10) os macOS Catalina 10.15.7 system x86_64, darwin19.6.0 ui unknown language (EN) collate en_US.UTF-8 ctype en_US.UTF-8 tz America/New_York date 2020-11-10 ─ Packages ─────────────────────────────────────────────────────────────────── package * version date lib source assertthat 0.2.1 2019-03-21 [1] RSPM (R 4.0.3) backports 1.2.0 2020-11-02 [1] RSPM (R 4.0.3) callr 3.5.1 2020-10-13 [1] RSPM (R 4.0.3) cli 2.1.0 2020-10-12 [1] RSPM (R 4.0.3) crayon 1.3.4 2017-09-16 [1] RSPM (R 4.0.3) desc 1.2.0 2018-05-01 [1] RSPM (R 4.0.3) devtools 2.3.2 2020-09-18 [1] CRAN (R 4.0.3) digest 0.6.27 2020-10-24 [1] RSPM (R 4.0.3) ellipsis 0.3.1 2020-05-15 [1] RSPM (R 4.0.3) fansi 0.4.1 2020-01-08 [1] RSPM (R 4.0.3) fs 1.5.0 2020-07-31 [1] RSPM (R 4.0.3) generics 0.1.0 2020-10-31 [1] RSPM (R 4.0.3) glue 1.4.2 2020-08-27 [1] RSPM (R 4.0.3) lubridate 1.7.9.9001 2020-11-10 [1] Github (tidyverse/lubridate@8071a1f) magrittr 1.5 2014-11-22 [1] RSPM (R 4.0.3) memoise 1.1.0 2017-04-21 [1] CRAN (R 4.0.3) pkgbuild 1.1.0 2020-07-13 [1] RSPM (R 4.0.3) pkgload 1.1.0 2020-05-29 [1] RSPM (R 4.0.3) prettyunits 1.1.1 2020-01-24 [1] RSPM (R 4.0.3) processx 3.4.4 2020-09-03 [1] RSPM (R 4.0.3) ps 1.4.0 2020-10-07 [1] RSPM (R 4.0.3) R6 2.5.0 2020-10-28 [1] RSPM (R 4.0.3) Rcpp 1.0.5 2020-07-06 [1] RSPM (R 4.0.3) remotes 2.2.0 2020-07-21 [1] CRAN (R 4.0.3) rlang 0.4.8 2020-10-08 [1] RSPM (R 4.0.3) rprojroot 1.3-2 2018-01-03 [1] RSPM (R 4.0.3) sessioninfo 1.1.1 2018-11-05 [1] CRAN (R 4.0.3) testthat 3.0.0 2020-10-31 [1] RSPM (R 4.0.3) usethis 1.6.3 2020-09-17 [1] CRAN (R 4.0.3) withr 2.3.0 2020-09-22 [1] RSPM (R 4.0.3) [1] /usr/local/lib/R/4.0/site-library [2] /usr/local/Cellar/r/4.0.3/lib/R/library ```

aylapear commented 3 years ago

The update to Lubridate fixed the issue for me! My macOS is up to date and has no updates pending (see session info for details). I tried with a random selection of different time zones and they all worked. 

> Sys.getenv("TZDIR")
[1] "macOS"
> devtools::install_github("tidyverse/lubridate")
> library(lubridate)
> x <- as.POSIXct("2000-01-01", tz = "America/Costa_Rica")
> year(x)
[1] 2000
> year(x) <-  1972
> year(x)
[1] 1972
> Sys.getenv("TZDIR")
[1] "/usr/share/zoneinfo"
Session Info ``` > devtools::session_info() ─ Session info ──────────────────────────────────────────────────────────────────────────────────────────── setting value version R version 4.0.3 (2020-10-10) os macOS Catalina 10.15.7 system x86_64, darwin17.0 ui RStudio language (EN) collate en_CA.UTF-8 ctype en_CA.UTF-8 tz America/Vancouver date 2020-11-12 ─ Packages ──────────────────────────────────────────────────────────────────────────────────────────────── ! package * version date lib source assertthat 0.2.1 2019-03-21 [1] CRAN (R 4.0.2) backports 1.2.0 2020-11-02 [1] CRAN (R 4.0.2) callr 3.5.1 2020-10-13 [1] CRAN (R 4.0.2) cli 2.1.0 2020-10-12 [1] CRAN (R 4.0.2) crayon 1.3.4 2017-09-16 [1] CRAN (R 4.0.2) curl 4.3 2019-12-02 [1] CRAN (R 4.0.1) desc 1.2.0 2018-05-01 [1] CRAN (R 4.0.2) devtools 2.3.2 2020-09-18 [1] CRAN (R 4.0.2) digest 0.6.27 2020-10-24 [1] CRAN (R 4.0.2) ellipsis 0.3.1 2020-05-15 [1] CRAN (R 4.0.2) fansi 0.4.1 2020-01-08 [1] CRAN (R 4.0.2) fs 1.5.0 2020-07-31 [1] CRAN (R 4.0.2) generics 0.1.0 2020-10-31 [1] CRAN (R 4.0.2) glue 1.4.2 2020-08-27 [1] CRAN (R 4.0.2) V lubridate * 1.7.9 2020-11-12 [1] Github (tidyverse/lubridate@8071a1f) magrittr 1.5 2014-11-22 [1] CRAN (R 4.0.2) memoise 1.1.0 2017-04-21 [1] CRAN (R 4.0.2) pkgbuild 1.1.0 2020-07-13 [1] CRAN (R 4.0.2) pkgload 1.1.0 2020-05-29 [1] CRAN (R 4.0.2) prettyunits 1.1.1 2020-01-24 [1] CRAN (R 4.0.2) processx 3.4.4 2020-09-03 [1] CRAN (R 4.0.2) ps 1.4.0 2020-10-07 [1] CRAN (R 4.0.2) R6 2.5.0 2020-10-28 [1] CRAN (R 4.0.2) Rcpp 1.0.5 2020-07-06 [1] CRAN (R 4.0.2) remotes 2.2.0 2020-07-21 [1] CRAN (R 4.0.2) rlang 0.4.8 2020-10-08 [1] CRAN (R 4.0.2) rprojroot 1.3-2 2018-01-03 [1] CRAN (R 4.0.2) rstudioapi 0.12 2020-11-10 [1] CRAN (R 4.0.3) sessioninfo 1.1.1 2018-11-05 [1] CRAN (R 4.0.2) testthat 3.0.0 2020-10-31 [1] CRAN (R 4.0.2) usethis 1.6.3 2020-09-17 [1] CRAN (R 4.0.2) withr 2.3.0 2020-09-22 [1] CRAN (R 4.0.2) ```
adamhsparks commented 3 years ago

Oddly, this went away but now it's back again.

Even more oddly, if I try to make a reprex, using reprex::reprex() it works properly in the resulting reprex, but I cannot make it work interactively.

LeeMendelowitz commented 3 years ago

@adamhsparks Try calling Sys.timezone() before loading the lubridate package. Does that help?

trifle commented 3 years ago

Perhaps as an additional datapoint, this bug is inconsistent across different timezones for me:

> lubridate::dmy_hm("1 Sep 2020 1:00pm", tz="Africa/Blantyre")
[1] "2020-09-01 13:00:00 CAT"
> lubridate::dmy_hm("1 Sep 2020 1:00pm", tz="Europe/London")
[1] "2020-09-01 13:00:00 BST"
> lubridate::dmy_hm("1 Sep 2020 1:00pm", tz="Europe/Berlin")
Error in C_force_tz(time, tz = tzone, roll) : 
  CCTZ: Unrecognized output timezone: "Europe/Berlin"

Some TZs work, while others don't, with no apparent pattern.

session info

``` > devtools::session_info() ─ Session info ────────────────────────────────────────────────────────────────── setting value version R version 4.0.3 (2020-10-10) os macOS Big Sur 10.16 system x86_64, darwin17.0 ui RStudio language (EN) collate en_US.UTF-8 ctype en_US.UTF-8 tz Europe/Berlin date 2020-11-25 ─ Packages ────────────────────────────────────────────────────────────────────── package * version date lib source assertthat 0.2.1 2019-03-21 [1] CRAN (R 4.0.2) backports 1.2.0 2020-11-02 [1] CRAN (R 4.0.2) broom 0.7.2 2020-10-20 [1] CRAN (R 4.0.2) callr 3.5.1 2020-10-13 [1] CRAN (R 4.0.2) cellranger 1.1.0 2016-07-27 [1] CRAN (R 4.0.2) cli 2.2.0 2020-11-20 [1] CRAN (R 4.0.2) colorspace 2.0-0 2020-11-11 [1] CRAN (R 4.0.3) crayon 1.3.4 2017-09-16 [1] CRAN (R 4.0.2) curl 4.3 2019-12-02 [1] CRAN (R 4.0.1) DBI 1.1.0 2019-12-15 [1] CRAN (R 4.0.2) dbplyr 2.0.0 2020-11-03 [1] CRAN (R 4.0.2) desc 1.2.0 2018-05-01 [1] CRAN (R 4.0.2) devtools 2.3.2 2020-09-18 [1] CRAN (R 4.0.2) digest 0.6.27 2020-10-24 [1] CRAN (R 4.0.2) dplyr * 1.0.2 2020-08-18 [1] CRAN (R 4.0.2) ellipsis 0.3.1 2020-05-15 [1] CRAN (R 4.0.2) fansi 0.4.1 2020-01-08 [1] CRAN (R 4.0.2) farver 2.0.3 2020-01-16 [1] CRAN (R 4.0.2) forcats * 0.5.0 2020-03-01 [1] CRAN (R 4.0.2) fs 1.5.0 2020-07-31 [1] CRAN (R 4.0.2) generics 0.1.0 2020-10-31 [1] CRAN (R 4.0.2) ggplot2 * 3.3.2 2020-06-19 [1] CRAN (R 4.0.2) glue 1.4.2 2020-08-27 [1] CRAN (R 4.0.2) gtable 0.3.0 2019-03-25 [1] CRAN (R 4.0.2) haven 2.3.1 2020-06-01 [1] CRAN (R 4.0.2) hms 0.5.3 2020-01-08 [1] CRAN (R 4.0.2) httr 1.4.2 2020-07-20 [1] CRAN (R 4.0.2) jsonlite * 1.7.1 2020-09-07 [1] CRAN (R 4.0.2) labeling 0.4.2 2020-10-20 [1] CRAN (R 4.0.2) lifecycle 0.2.0 2020-03-06 [1] CRAN (R 4.0.2) lubridate * 1.7.9.9001 2020-11-25 [1] Github (tidyverse/lubridate@6c535c8) magrittr 2.0.1 2020-11-17 [1] CRAN (R 4.0.3) memoise 1.1.0 2017-04-21 [1] CRAN (R 4.0.2) modelr 0.1.8 2020-05-19 [1] CRAN (R 4.0.2) munsell 0.5.0 2018-06-12 [1] CRAN (R 4.0.2) pillar 1.4.7 2020-11-20 [1] CRAN (R 4.0.3) pkgbuild 1.1.0 2020-07-13 [1] CRAN (R 4.0.2) pkgconfig 2.0.3 2019-09-22 [1] CRAN (R 4.0.2) pkgload 1.1.0 2020-05-29 [1] CRAN (R 4.0.2) prettyunits 1.1.1 2020-01-24 [1] CRAN (R 4.0.2) processx 3.4.4 2020-09-03 [1] CRAN (R 4.0.2) ps 1.4.0 2020-10-07 [1] CRAN (R 4.0.2) purrr * 0.3.4 2020-04-17 [1] CRAN (R 4.0.2) R.cache 0.14.0 2019-12-06 [1] CRAN (R 4.0.2) R.methodsS3 1.8.1 2020-08-26 [1] CRAN (R 4.0.2) R.oo 1.24.0 2020-08-26 [1] CRAN (R 4.0.2) R.utils 2.10.1 2020-08-26 [1] CRAN (R 4.0.2) R6 2.5.0 2020-10-28 [1] CRAN (R 4.0.2) Rcpp 1.0.5 2020-07-06 [1] CRAN (R 4.0.2) readr * 1.4.0 2020-10-05 [1] CRAN (R 4.0.2) readxl 1.3.1 2019-03-13 [1] CRAN (R 4.0.2) rematch2 2.1.2 2020-05-01 [1] CRAN (R 4.0.2) remotes 2.2.0 2020-07-21 [1] CRAN (R 4.0.2) reprex 0.3.0 2019-05-16 [1] CRAN (R 4.0.2) rlang 0.4.8 2020-10-08 [1] CRAN (R 4.0.2) rprojroot 2.0.2 2020-11-15 [1] CRAN (R 4.0.3) rsconnect 0.8.16 2019-12-13 [1] CRAN (R 4.0.2) rstudioapi 0.13 2020-11-12 [1] CRAN (R 4.0.3) rvest 0.3.6 2020-07-25 [1] CRAN (R 4.0.2) scales 1.1.1 2020-05-11 [1] CRAN (R 4.0.2) sessioninfo 1.1.1 2018-11-05 [1] CRAN (R 4.0.2) stringi 1.5.3 2020-09-09 [1] CRAN (R 4.0.2) stringr * 1.4.0 2019-02-10 [1] CRAN (R 4.0.2) styler * 1.3.2 2020-02-23 [1] CRAN (R 4.0.2) testthat 3.0.0 2020-10-31 [1] CRAN (R 4.0.2) tibble * 3.0.4 2020-10-12 [1] CRAN (R 4.0.2) tidyr * 1.1.2 2020-08-27 [1] CRAN (R 4.0.2) tidyselect 1.1.0 2020-05-11 [1] CRAN (R 4.0.2) tinytex 0.27 2020-11-01 [1] CRAN (R 4.0.2) usethis 1.6.3 2020-09-17 [1] CRAN (R 4.0.2) utf8 1.1.4 2018-05-24 [1] CRAN (R 4.0.2) vctrs 0.3.5 2020-11-17 [1] CRAN (R 4.0.3) withr 2.3.0 2020-09-22 [1] CRAN (R 4.0.2) xfun 0.19 2020-10-30 [1] CRAN (R 4.0.2) xml2 1.3.2 2020-04-23 [1] CRAN (R 4.0.2) [1] /Library/Frameworks/R.framework/Versions/4.0/Resources/library ```

rrohwer commented 3 years ago

Hi everyone, Just wanted to let you know that this bug appeared after a mac update for me, not an R update, in case that helps you with your troubleshooting. (Using R 4.0.3) The mac update was a security update that took me to Catalina 10.15.7. I solved it with the Sys.setenv("TZDIR" = "/Library/Frameworks/R.framework/Resources/share/zoneinfo/") command that @gaborcsardi suggested. Thanks! Robin

mmuurr commented 3 years ago

Since feedback was asked-for, I can report that the most recent commit (aab2e30) does not resolve the issue for me 'out-of-the-box'. R 4.0.3, Mac OS 10.15.7.

And when I try the explicit TZDIR setting I can resolve it, but it requires knowing the explicit path (which required a manual search through /usr/local). Since I used brew as the installation mechanism (which perhaps was a poor choice?), on my machine this solves the 

Sys.setenv("TZDIR" = "/usr/local/Cellar/r/4.0.3_2/lib/R/share/zoneinfo/")

Though I'm not sure how to easily detect that path a priori.

hadley commented 3 years ago

@mmuurr what does file.exists(file.path(R.home("share"), "zoneinfo")) return?

mmuurr commented 3 years ago

@hadley

file.exists(file.path(R.home("share"), "zoneinfo"))
# [1] TRUE

... but also I'm starting to question my sanity as I'm getting errors/warnings in different projects, but with all the same package versions installed. So apologies in advance for the long post here, but I'm hoping this helps track down some of the inconsistencies various people have observed.

Below are four cases:

In all cases:

An renv project

Within RStudio

For starters, my reprex in RStudio (IDE) is failing to produce the same warning that appears in the RStudio console when executing the exact same script by hand :-/. Here's a reprex::reprex()-ed script without any TZ warning/error:

.libPaths()
#> [1] "/Users/my_redacted_project_dirpath/renv/library/R-4.0/x86_64-apple-darwin19.6.0"
#> [2] "/private/var/folders/xl/jg5dprqs2sxcbqq9hm78b9xr0000gn/T/Rtmpm9gJga/renv-system-library"      
#> [3] "/private/var/folders/xl/jg5dprqs2sxcbqq9hm78b9xr0000gn/T/Rtmpdx3RgV/renv-system-library"

file.path(R.home("share"))
#> [1] "/usr/local/Cellar/r/4.0.3_2/lib/R/share"

file.exists(file.path(R.home("share"), "zoneinfo"))
#> [1] TRUE

Sys.getenv("TZDIR")
#> [1] "macOS"

lubridate::now()
#> [1] "2021-01-19 13:26:23 MST"

sessionInfo()
#> R version 4.0.3 (2020-10-10)
#> Platform: x86_64-apple-darwin19.6.0 (64-bit)
#> Running under: macOS Catalina 10.15.7
#> 
#> Matrix products: default
#> BLAS:   /usr/local/Cellar/openblas/0.3.13/lib/libopenblasp-r0.3.13.dylib
#> LAPACK: /usr/local/Cellar/r/4.0.3_2/lib/R/lib/libRlapack.dylib
#> 
#> locale:
#> [1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
#> 
#> attached base packages:
#> [1] stats     graphics  grDevices datasets  utils     methods   base     
#> 
#> loaded via a namespace (and not attached):
#>  [1] Rcpp_1.0.5        lubridate_1.7.9.2 digest_0.6.27     magrittr_2.0.1   
#>  [5] evaluate_0.14     highr_0.8         rlang_0.4.10      stringi_1.5.3    
#>  [9] renv_0.11.0       generics_0.1.0    rmarkdown_2.6     tools_4.0.3      
#> [13] stringr_1.4.0     xfun_0.19         yaml_2.2.1        compiler_4.0.3   
#> [17] htmltools_0.5.0   knitr_1.30
Created on 2021-01-19 by the reprex package (v0.3.0)

Note that:

  1. this project happens to be renv-ed
  2. Sys.getenv("TZDIR") yields "macOS"
  3. lubridate::now() has no error or warning.

Now, immediately after running that script, at the RStudio console I try lubridate::now(), I get:

lubridate::now()
# [1] "2021-01-19 13:26:34 MST"
# Warning message:
# In with_tz(Sys.time(), tzone) : Unrecognized time zone ''

Outside of RStudio

When the same script is from the shell via R --no-save < reprex.R:

> .libPaths()
[1] "/Users/my_redacted_project_dirpath/renv/library/R-4.0/x86_64-apple-darwin19.6.0"
[2] "/private/var/folders/xl/jg5dprqs2sxcbqq9hm78b9xr0000gn/T/RtmpyuA8YQ/renv-system-library"

> file.path(R.home("share"))
[1] "/usr/local/Cellar/r/4.0.3_2/lib/R/share"

> file.exists(file.path(R.home("share"), "zoneinfo"))
[1] TRUE

> Sys.getenv("TZDIR")
[1] ""

> lubridate::now()
[1] "2021-01-19 13:44:43 MST"
Warning message:
In with_tz(Sys.time(), tzone) : Unrecognized time zone ''

> sessionInfo()
R version 4.0.3 (2020-10-10)
Platform: x86_64-apple-darwin19.6.0 (64-bit)
Running under: macOS Catalina 10.15.7

Matrix products: default
BLAS:   /usr/local/Cellar/openblas/0.3.13/lib/libopenblasp-r0.3.13.dylib
LAPACK: /usr/local/Cellar/r/4.0.3_2/lib/R/lib/libRlapack.dylib

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats     graphics  grDevices datasets  utils     methods   base

loaded via a namespace (and not attached):
[1] compiler_4.0.3    generics_0.1.0    tools_4.0.3       Rcpp_1.0.5
[5] lubridate_1.7.9.2 renv_0.11.0
  1. The TZ warning is present.
  2. Sys.getenv("TZDIR") now yields "" (instead of "macOS" when run in RStudio).
  3. Fewer packages are loaded, but of those that are loaded, the versions are all identical between the Rstudio and non-Rstudio cases.
  4. The R version is identical (4.0.3).

Non-renv

Within RStudio & Outside of RStudio

To save some space, I've just combined the results of the non-renv case, as the only difference in the within-RStudio and outside-of-RStudio scenarios is the list of loaded packages, but in both cases there are no TZ warnings/errors.

reprex::reprex() output from within RStudio (for comparison to the above scenarios):

.libPaths()
#> [1] "/usr/local/lib/R/4.0/site-library"        
#> [2] "/usr/local/Cellar/r/4.0.3_2/lib/R/library"

file.path(R.home("share"))
#> [1] "/usr/local/Cellar/r/4.0.3_2/lib/R/share"

file.exists(file.path(R.home("share"), "zoneinfo"))
#> [1] TRUE

Sys.getenv("TZDIR")
#> [1] "macOS"

lubridate::now()
#> [1] "2021-01-19 13:51:09 MST"

sessionInfo()
#> R version 4.0.3 (2020-10-10)
#> Platform: x86_64-apple-darwin19.6.0 (64-bit)
#> Running under: macOS Catalina 10.15.7
#> 
#> Matrix products: default
#> BLAS:   /usr/local/Cellar/openblas/0.3.13/lib/libopenblasp-r0.3.13.dylib
#> LAPACK: /usr/local/Cellar/r/4.0.3_2/lib/R/lib/libRlapack.dylib
#> 
#> locale:
#> [1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
#> 
#> attached base packages:
#> [1] stats     graphics  grDevices utils     datasets  methods   base     
#> 
#> loaded via a namespace (and not attached):
#>  [1] compiler_4.0.3    magrittr_2.0.1    generics_0.1.0    tools_4.0.3      
#>  [5] htmltools_0.5.0   yaml_2.2.1        Rcpp_1.0.5        lubridate_1.7.9.2
#>  [9] stringi_1.5.3     rmarkdown_2.6     highr_0.8         knitr_1.30       
#> [13] stringr_1.4.0     xfun_0.20         digest_0.6.27     rlang_0.4.10     
#> [17] evaluate_0.14
Created on 2021-01-19 by the reprex package (v0.3.0)

And if I run lubridate::now() at the console (both within- and outside-of- RStudio), no warning.

I realize all of that is quite long but I figured 'err on the side of verbosity' in an effort to provide any/all helpful clues :-/

Cheers!

hadley commented 3 years ago

@mmuurr since your problem seems a bit different can you please file a new issue?

andybyte commented 3 years ago

Try calling Sys.timezone() before loading the lubridate package. Does that help? @LeeMendelowitz

This helped me. Just had this issue recently and I don't remember what changed in my system except i did update packages. Not sure if Lubridate was one of them but if I reset the R environment and first run Sys.timezone that seems to do the trick.