DateTime implementation on linux, dates before year 2020 return incorrect timezone

insinfo commented 3 months ago

I'm facing a problem, when I define a date as year: 2000, day: 1, month: 1, hour: 0, minute: 0, millisecond: 0 and microsecond: 0, in windows with timezone America / Sao Paulo when I convert it to string I receive the correct value "2000-01-01 00:00:00.000 -3", but in linux with timezone America / Sao Paulo I receive "2000-01-01 00:00:00.000 -2" it is bringing with timestamp -2 which is incorrect, it was supposed to receive -3 too, this is causing a problem in a postgresql driver implementation that receives for example the value microseconds 774702600000000 which is the local time "2024-07-19 11:10:00" from the postgresql driver and when I convert it to DateTime in dart gets wrong with one hour difference on linux.


void main(List<String> args) async {
  //774702600000000 = 2024-07-19 11:10:00 = DateTime(2024, 07, 19, 11, 10, 00)
  final dur = Duration(microseconds: 774702600000000);
  print('Duration $dur ${dur.inDays}');
  final dtUtc = DateTime.utc(2000).add(dur);

  final dtLocalDecode = DateTime(2000).add(dur);
  final dtLocal = dtUtc.toLocal();
  final dartDt = DateTime(2000, 1, 1, 0, 0, 0, 0, 0);
  final dartNow = DateTime.now();
  print('dtUtc $dtUtc ${dtUtc.timeZoneOffset}  ${dtUtc.timeZoneName}');
  print(
      'dtLocal utcToLocal $dtLocal ${dtLocal.timeZoneOffset}  ${dtLocal.timeZoneName}');
  print(
      'dtLocal decode $dtLocalDecode ${dtLocalDecode.timeZoneOffset}  ${dtLocalDecode.timeZoneName}');
  print('dartDt  $dartDt ${dartDt.timeZoneOffset}  ${dartDt.timeZoneName}');
  print('dartNow  $dartNow ${dartNow.timeZoneOffset}  ${dartNow.timeZoneName}');  
}

result in Windows 11

Duration 215195:10:00.000000 8966
dtUtc 2024-07-19 11:10:00.000Z 0:00:00.000000  UTC
dtLocal utcToLocal 2024-07-19 08:10:00.000 -3:00:00.000000  Hora oficial do Brasil
dtLocal decode 2024-07-19 11:10:00.000 -3:00:00.000000  Hora oficial do Brasil
dartDt  2000-01-01 00:00:00.000 -3:00:00.000000  Hora oficial do Brasil
dartNow  2024-07-24 15:54:01.433210 -3:00:00.000000  Hora oficial do Brasil

result in Ubuntu 22.04.2 LTS

Duration 215195:10:00.000000 8966
dtUtc 2024-07-19 11:10:00.000Z 0:00:00.000000  UTC
dtLocal utcToLocal 2024-07-19 08:10:00.000 -3:00:00.000000  -03
dtLocal decode 2024-07-19 10:10:00.000 -3:00:00.000000  -03
dartDt  2000-01-01 00:00:00.000 -2:00:00.000000  -02
dartNow  2024-07-24 15:53:50.647503 -3:00:00.000000  -03

Note that dtLocal decode on Windows displays 11 hours and on Linux 10 hours.

Also note that dartDt on Windows displays timezone -3 and on Linux -2.

dart-github-bot commented 3 months ago

Summary: The DateTime implementation on Linux incorrectly handles time zones for dates before 2020, resulting in an off-by-one-hour discrepancy when converting from UTC to local time. This issue affects the DateTime constructor and toLocal() method, leading to incorrect time zone offsets for dates in the past.

leonardomw commented 3 months ago

I'm also having this problem and I didn't find anything in the documentation that differentiates DateTime on Linux from Windows.

insinfo commented 3 months ago

from what I saw this problem also happens in Debian GNU/Linux 10, from what I can find out it seems that in Linux the DateTime implementation is taking an old timezone transition and not the current TimeZone different from the Windows implementation

lrhn commented 3 months ago

If time-zone information differs between Windows and Linux, the difference is likely in the operating systems. Windows is known for not having all older time-zone information available in some time zones. I don't know if that's the case here.

maciel-neto commented 3 months ago

How to create a DateTime instance in Dart using the current timezone in Linux so that the behavior is similar to what happens in Windows.

insinfo commented 3 months ago

@lrhn In Java the behavior is identical in both Windows and Linux, see that dtLocal decode displays as 11:10 on windows and linux, which was the behavior I expected for dart

public class Main {
    public static void main(String[] args) {
        // 774702600000000 microseconds = 2024-07-19 11:10:00
        Duration duration = Duration.of(774702600000000L, ChronoUnit.MICROS);
        System.out.println("Duration " + duration + " " + duration.toDays());

        ZonedDateTime dtUtc = ZonedDateTime.of(LocalDateTime.of(2000, 1, 1, 0, 0), ZoneId.of("UTC")).plus(duration);
        LocalDateTime dtLocalDecode = LocalDateTime.of(2000, 1, 1, 0, 0).plus(duration);
        ZonedDateTime dtLocal = dtUtc.withZoneSameInstant(ZoneId.systemDefault());
        LocalDateTime dartDt = LocalDateTime.of(2000, 1, 1, 0, 0);
        LocalDateTime dartNow = LocalDateTime.now();

        System.out.println("dtUtc " + dtUtc + " " + dtUtc.getOffset() + "  " + dtUtc.getZone());
        System.out.println("dtLocal utcToLocal " + dtLocal + " " + dtLocal.getOffset() + "  " + dtLocal.getZone());
        System.out.println("dtLocal decode " + dtLocalDecode + " " + ZoneId.systemDefault().getRules().getOffset(dtLocalDecode) + "  " + ZoneId.systemDefault());
        System.out.println("dartDt  " + dartDt + " " + ZoneId.systemDefault().getRules().getOffset(dartDt) + "  " + ZoneId.systemDefault());
        System.out.println("dartNow  " + dartNow + " " + ZoneId.systemDefault().getRules().getOffset(dartNow) + "  " + ZoneId.systemDefault());
    }
}

windows

Microsoft Windows 11 Pro 10.0.22631 64 bits
javac Main.java; java Main
Duration PT215195H10M 8966
dtUtc 2024-07-19T11:10Z[UTC] Z  UTC
dtLocal utcToLocal 2024-07-19T08:10-03:00[America/Sao_Paulo] -03:00  America/Sao_Paulo
dtLocal decode 2024-07-19T11:10 -03:00  America/Sao_Paulo
dartDt  2000-01-01T00:00 -02:00  America/Sao_Paulo
dartNow  2024-07-25T15:52:53.860442900 -03:00  America/Sao_Paulo

linux

Ubuntu 22.04.2 LTS
 javac Main.java; java Main
Duration PT215195H10M 8966
dtUtc 2024-07-19T11:10Z[UTC] Z  UTC
dtLocal utcToLocal 2024-07-19T08:10-03:00[America/Sao_Paulo] -03:00  America/Sao_Paulo
dtLocal decode 2024-07-19T11:10 -03:00  America/Sao_Paulo
dartDt  2000-01-01T00:00 -02:00  America/Sao_Paulo
dartNow  2024-07-25T15:52:59.152457 -03:00  America/Sao_Paulo

insinfo commented 3 months ago

In C# the behavior is also identical on Windows and Linux, in both the dtLocal decode is 11:10

class Program
{
    static void Main(string[] args)
    {
        // 774702600000000 microseconds = 2024-07-19 11:10:00 = DateTime(2024, 07, 19, 11, 10, 00)
        TimeSpan dur = TimeSpan.FromTicks(774702600000000 * 10); // Convert microseconds to ticks 
        Console.WriteLine($"Duration {dur} {dur.Days}");

        DateTime dtUtc = new DateTime(2000, 1, 1, 0, 0, 0, DateTimeKind.Utc).Add(dur);
        DateTime dtLocalDecode = new DateTime(2000, 1, 1, 0, 0, 0, DateTimeKind.Local).Add(dur);
        DateTime dtLocal = dtUtc.ToLocalTime();
        DateTime dartDt = new DateTime(2000, 1, 1, 0, 0, 0, DateTimeKind.Local);
        DateTime dartNow = DateTime.Now;

        Console.WriteLine($"dtUtc {dtUtc}  {dtUtc.Kind}");
        Console.WriteLine($"dtLocal utcToLocal {dtLocal}   {dtLocal.Kind}");
        Console.WriteLine($"dtLocal decode {dtLocalDecode}  {dtLocalDecode.Kind}");
        Console.WriteLine($"dartDt  {dartDt}   {dartDt.Kind}");
        Console.WriteLine($"dartNow  {dartNow}   {dartNow.Kind}");
    }
}

windows

dotnet run
Duration 8966.11:10:00 8966
dtUtc 19/07/2024 11:10:00  Utc
dtLocal utcToLocal 19/07/2024 08:10:00   Local
dtLocal decode 19/07/2024 11:10:00  Local
dartDt  01/01/2000 00:00:00   Local
dartNow  25/07/2024 16:17:19   Local

linux

 dotnet run
Duration 8966.11:10:00 8966
dtUtc 19/07/2024 11:10:00  Utc
dtLocal utcToLocal 19/07/2024 08:10:00   Local
dtLocal decode 19/07/2024 11:10:00  Local
dartDt  01/01/2000 00:00:00   Local
dartNow  25/07/2024 16:18:55   Local

insinfo commented 3 months ago

@lrhn My use case is in this PostgreSQL driver, when decoding data of type timestamp without timezone and type date, since they are not UTC. For these fields, you want the value sent to be returned from the db in an identical form without alteration. In most cases, they will be local using the same timezone as the current server where PostgreSQL runs and runs the backend application to do comparisons and operations with DateTime.now().

    case PostgreSQLDataType.date:
        final value = buffer.getInt32(0);
        //infinity || -infinity
        if (value == 2147483647 || value == -2147483648) {
          return null;
        }
        if (timeZone.forceDecodeDateAsUTC) {
          return DateTime.utc(2000).add(Duration(days: value)) as T;
        }

        // https://github.com/dart-lang/sdk/issues/56312
        // ignore past timestamp transitions and use only current timestamp in local datetime        
        final nowDt = DateTime.now();
        var baseDt = DateTime(2000);
        if (baseDt.timeZoneOffset != nowDt.timeZoneOffset) {
          final difference = baseDt.timeZoneOffset - nowDt.timeZoneOffset;
          baseDt = baseDt.add(difference);
        }
        return baseDt.add(Duration(days: value)) as T;

      case PostgreSQLDataType.timestampWithoutTimezone:
        final value = buffer.getInt64(0);
        //infinity || -infinity
        if (value == 9223372036854775807 || value == -9223372036854775808) {
          return null;
        }
        if (timeZone.forceDecodeTimestampAsUTC) {
          return DateTime.utc(2000).add(Duration(microseconds: value)) as T;
        }

        // https://github.com/dart-lang/sdk/issues/56312
        // ignore previous timestamp transitions and use only the current system timestamp in local date and time so that the behavior is correct on Windows and Linux
        final nowDt = DateTime.now();
        var baseDt = DateTime(2000);
        if (baseDt.timeZoneOffset != nowDt.timeZoneOffset) {
          final difference = baseDt.timeZoneOffset - nowDt.timeZoneOffset;
          baseDt = baseDt.add(difference);
        }
        return baseDt.add(Duration(microseconds: value)) as T;

      case PostgreSQLDataType.timestampWithTimezone:
        final value = buffer.getInt64(0);

        //infinity || -infinity
        if (value == 9223372036854775807 || value == -9223372036854775808) {
          return null;
        }

        var datetime = DateTime.utc(2000).add(Duration(microseconds: value));
        if (timeZone.forceDecodeTimestamptzAsUTC) {
          return datetime as T;
        }
        if (timeZone.value.toLowerCase() == 'utc') {
          return datetime as T;
        }

        final pgTimeZone = timeZone.value.toLowerCase();
        final tzLocations = tz.timeZoneDatabase.locations.entries
            .where((e) {
              return (e.key.toLowerCase() == pgTimeZone ||
                  e.value.currentTimeZone.abbreviation.toLowerCase() ==
                      pgTimeZone);
            })
            .map((e) => e.value)
            .toList();

        if (tzLocations.isEmpty) {
          throw tz.LocationNotFoundException(
              'Location with the name "$pgTimeZone" doesn\'t exist');
        }
        final tzLocation = tzLocations.first;
        //define location for TZDateTime.toLocal()
        tzenv.setLocalLocation(tzLocation);

        final offsetInMilliseconds = tzLocation.currentTimeZone.offset;
        // Conversion of milliseconds to hours
        final double offset = offsetInMilliseconds / (1000 * 60 * 60);

        if (offset < 0) {
          final subtr = Duration(
              hours: offset.abs().truncate(),
              minutes: ((offset.abs() % 1) * 60).round());
          datetime = datetime.subtract(subtr);
          final specificDate = tz.TZDateTime(
              tzLocation,
              datetime.year,
              datetime.month,
              datetime.day,
              datetime.hour,
              datetime.minute,
              datetime.second,
              datetime.millisecond,
              datetime.microsecond);
          return specificDate as T;
        } else if (offset > 0) {
          final addr = Duration(
              hours: offset.truncate(), minutes: ((offset % 1) * 60).round());
          datetime = datetime.add(addr);
          final specificDate = tz.TZDateTime(
              tzLocation,
              datetime.year,
              datetime.month,
              datetime.day,
              datetime.hour,
              datetime.minute,
              datetime.second,
              datetime.millisecond,
              datetime.microsecond);
          return specificDate as T;
        }

        return datetime as T;

https://github.com/insinfo/postgres_fork/blob/master/lib/src/binary_codec.dart

dart-lang / sdk