Wrong date when formatting DateTime with format string

[mnussbaumer]

Hi, first thanks for all the cldr packages that you maintain, they are very useful. I think I found a bug - I was getting the wrong date when formatting a date time. So all the following dates in 2027, from Dec. 27th onwards, are wrongly formatted as 2028. But other years don't seem to produce the same issues.

The following script exemplifies the issue:

Mix.install([
  {:jason, "~> 1.2"},
  {:ex_cldr, "~> 2.28"},
  {:ex_cldr_dates_times, "~> 2.0"}
])

defmodule Test.Cldr do
  use Cldr,
    locales: ["en"],
    default_locale: "en",
    providers: [Cldr.Number, Cldr.Calendar, Cldr.DateTime],
    json_library: Jason
end

[
  ~U[2027-12-23 05:00:40Z],
  ~U[2027-12-24 03:30:00Z],
  ~U[2027-12-25 05:00:00Z],
  ~U[2027-12-26 05:00:00Z],
  ~U[2027-12-27 05:00:23Z],
  ~U[2027-12-28 05:40:00Z],
  ~U[2027-12-29 12:00:00Z],
  ~U[2027-12-30 03:00:00Z],
  ~U[2028-12-29 12:00:00Z],
  ~U[2028-12-30 03:00:00Z]
]
|> Enum.each(fn datetime ->
  formatted = Test.Cldr.DateTime.to_string!(datetime, format: "MMM d, YYYY")
  IO.inspect(datetime: datetime, formatted: formatted)
end)

When I run this I get:

work-base@work-base:~/work$ mix run cldr_bug_case.exs --no-mix-exs
Generating Test.Cldr for 2 locales named [:en, :und] with a default locale named :en
[datetime: ~U[2027-12-23 05:00:40Z], formatted: "Dec 23, 2027"]
[datetime: ~U[2027-12-24 03:30:00Z], formatted: "Dec 24, 2027"]
[datetime: ~U[2027-12-25 05:00:00Z], formatted: "Dec 25, 2027"]
[datetime: ~U[2027-12-26 05:00:00Z], formatted: "Dec 26, 2027"]
[datetime: ~U[2027-12-27 05:00:23Z], formatted: "Dec 27, 2028"]
[datetime: ~U[2027-12-28 05:40:00Z], formatted: "Dec 28, 2028"]
[datetime: ~U[2027-12-29 12:00:00Z], formatted: "Dec 29, 2028"]
[datetime: ~U[2027-12-30 03:00:00Z], formatted: "Dec 30, 2028"],
[datetime: ~U[2028-12-29 12:00:00Z], formatted: "Dec 29, 2028"]
[datetime: ~U[2028-12-30 03:00:00Z], formatted: "Dec 30, 2028"]
work-base@work-base:~/work$ 

The weird thing is that this only happens for, in this case, the dates starting at December 27th, and only for the year 2027. The same dates with just the year 2028, or 29, all seem to work correctly on all dates.

edit ------

It's actually interesting I think there might be some days/year durations being calculated wrong somewhere. If we change the year for 2026, then the same happens but starts only on the 28th of Dec instead of 27th, if we change the year to 2025 then it starts only on the 29th, on 2024 only on the 30th, and 2023 doesn't.

-------------

I haven't got much experience with this library nor dates/calendars/dates formatting, nor how it arrives at the formatting but I was able to confirm that up to Cldr.DateTime.to_string it all is fine in terms of params and it is in the last format_backend.format that the wrong transformation occurs, adding some IO.inspects to the ex_cldr.datetime deps resulted in this:

MyApp.Cldr.DateTime.to_string(~U[2027-12-27 05:00:00Z], format: "MMM d, YYYY")
[
  backend: MyApp.Cldr,
  options: [
    number_system: :latn,
    style: :default,
    time_format: nil,
    date_format: nil,
    format: "MMM d, YYYY",
    locale: #Cldr.LanguageTag<en [validated]>
  ],
  format_string: "MMM d, YYYY",
  datetime: ~U[2027-12-27 05:00:00Z Cldr.Calendar.Gregorian],
  locale: #Cldr.LanguageTag<en [validated]>,
  formatted: "Dec 27, 2028"
]

So the date time at this calling point is correct, but the formatted version of it is 2028 instead of 2027. Hope this helps figuring it out when you have the time and patience, happy holidays,

Additional Info:

erlang 26.1.2
elixir 1.15.7-otp-26

work-base@work-base:~/work$ mix run cldr_bug_case.exs --no-mix-exs
Resolving Hex dependencies...
Resolution completed in 0.098s
New:
  cldr_utils 2.24.2
  decimal 2.1.1
  digital_token 0.6.0
  ex_cldr 2.37.5
  ex_cldr_calendars 1.23.0
  ex_cldr_currencies 2.15.1
  ex_cldr_dates_times 2.16.0
  ex_cldr_numbers 2.32.3
  Jason 1.4.1

[kipcole9]

Ouch, that sure doesn't look right. I'll be back at my computer in a couple of hours and will dig into this right away.

[kipcole9]

Ahhhh, the issue here is the the correct year format placeholder is y (lower case). The format placeholder Y is the ISO Week based year which accounts for the difference. The last few days of the Gregorian 2027 are in fact in the ISO Week year of 2028.

Using the y symbol the return value are correct:

iex> [
...>   ~U[2027-12-23 05:00:40Z],
...>   ~U[2027-12-24 03:30:00Z],
...>   ~U[2027-12-25 05:00:00Z],
...>   ~U[2027-12-26 05:00:00Z],
...>   ~U[2027-12-27 05:00:23Z],
...>   ~U[2027-12-28 05:40:00Z],
...>   ~U[2027-12-29 12:00:00Z],
...>   ~U[2027-12-30 03:00:00Z],
...>   ~U[2028-12-29 12:00:00Z],
...>   ~U[2028-12-30 03:00:00Z]
...> ]
iex> |> Enum.each(fn datetime ->
...>   formatted = MyApp.Cldr.DateTime.to_string!(datetime, format: "MMM d, yyyy")
...>   IO.inspect(datetime: datetime, formatted: formatted)
...> end)
[datetime: ~U[2027-12-23 05:00:40Z], formatted: "Dec 23, 2027"]
[datetime: ~U[2027-12-24 03:30:00Z], formatted: "Dec 24, 2027"]
[datetime: ~U[2027-12-25 05:00:00Z], formatted: "Dec 25, 2027"]
[datetime: ~U[2027-12-26 05:00:00Z], formatted: "Dec 26, 2027"]
[datetime: ~U[2027-12-27 05:00:23Z], formatted: "Dec 27, 2027"]
[datetime: ~U[2027-12-28 05:40:00Z], formatted: "Dec 28, 2027"]
[datetime: ~U[2027-12-29 12:00:00Z], formatted: "Dec 29, 2027"]
[datetime: ~U[2027-12-30 03:00:00Z], formatted: "Dec 30, 2027"]
[datetime: ~U[2028-12-29 12:00:00Z], formatted: "Dec 29, 2028"]
[datetime: ~U[2028-12-30 03:00:00Z], formatted: "Dec 30, 2028"]
:ok

The documentation for format codes does differentiate. But it can certainly use improvement and I'll work on the over the next couple of days.

[mnussbaumer]

@kipcole9 thank you so much for looking into it right away. I didn't fully understand when I read the formatting rules I just thought that the ISO one would be the most "universal" but yeah, ISOweek the week part should have given away it was something with specific behaviour - but because it had been working just fine for a long time and all the tests... Anyway thank you! I confirmed it works. I will update all formatting to use the yyyy variant and also look into the documentation to see if there's any point in adding some note in the formatting table.

Happy new year!