Fix date module - Githubissues

CrazyIvan359 commented 5 years ago

This PR makes several fixes and improvements to the date module:

Fix int conversion error when converting from Python datetime.datetime
Fix inverted timezone when converting from java.util.Date
Correct return type of xx_between methods from int to long
Change names of imported types to use package name prefixes for clarity
Make openHAB 1.x DateTimeType import optional (Fixes #190) in case compat1.x bundle is not installed

Signed-off-by: Michael Murton 6764025+CrazyIvan359@users.noreply.github.com

5iver commented 5 years ago

Could you please post your test script? It would save me some time.

CrazyIvan359 commented 5 years ago

Commented lines are for testing without compat1.x, uncomment them to test legacy datatype conversions. Script puts each data type through format_date starting from a ZonedDateTime and using available conversion functions, otherwise data constructor directly. format_date puts the passed date through to_java_datetime before applying formatting, so that tests in the other direction. A second round of the same tests but shifting the timezone to CDT, which shows a failing of java.util.Date where you cannot specify the timezone because it forcefully uses the local tz. And the last batch tests the delta functions with +5 and -5 units of the respective units.

from core.rules import rule
from core.triggers import when

import core.date
reload(core.date)
from core.date import *

from java.time import ZoneId, LocalDateTime
from org.joda.time import DateTime as JodaDateTime
from datetime import datetime as PyDateTime
from java.util import Date as javautilDate
from org.eclipse.smarthome.core.library.types import DateTimeType as eclipseDT
#from org.openhab.core.library.types import DateTimeType as legacyDT

@rule("test")
@when("Time cron */15 * * * * ?")
def testdate(event):
  testdate.log.warn("Local timezone")
  javatimeZDT = javaDateTime.now()
  testdate.log.warn("java.time.ZonedDateTime.now(): " + format_date(javatimeZDT))
  testdate.log.warn("java.time.LocalDateTime():     " + format_date(LocalDateTime.parse(format_date(javatimeZDT, "yyyy-MM-dd'T'HH:mm:ss"))))
  testdate.log.warn("to_joda_datetime():            " + format_date(to_joda_datetime(javatimeZDT)))
  testdate.log.warn("to_python_datetime():          " + format_date(to_python_datetime(javatimeZDT)))
  testdate.log.warn("to_java_calendar():            " + format_date(to_java_calendar(javatimeZDT)))
  testdate.log.warn("format_date(javautilDate):     " + format_date(javautilDate(javatimeZDT.toInstant().toEpochMilli())))
  testdate.log.warn("format_date(eclipseDT):        " + format_date(eclipseDT(format_date(javatimeZDT))))
  #testdate.log.warn("format_date(legacyDT):         " + format_date(legacyDT(format_date(javatimeZDT))))

  testdate.log.warn("CDT timezone")
  javatimeZDT = javatimeZDT.withZoneSameInstant(ZoneId.of("America/Chicago"))
  testdate.log.warn("java.time.ZonedDateTime.now(): " + format_date(javatimeZDT))
  testdate.log.warn("to_joda_datetime():            " + format_date(to_joda_datetime(javatimeZDT)))
  testdate.log.warn("to_python_datetime():          " + format_date(to_python_datetime(javatimeZDT)))
  testdate.log.warn("to_java_calendar():            " + format_date(to_java_calendar(javatimeZDT)))
  testdate.log.warn("format_date(javautilDate):     " + format_date(javautilDate(javatimeZDT.toInstant().toEpochMilli())))
  testdate.log.warn("format_date(eclipseDT):        " + format_date(eclipseDT(format_date(javatimeZDT))))
  #testdate.log.warn("format_date(legacyDT):         " + format_date(legacyDT(format_date(javatimeZDT, "yyyy-MM-dd'T'HH:mm:ssz"))))

  testdate.log.warn("Differential methods")
  javatimeZDT = javaDateTime.now()
  testdate.log.warn("days_between(+5):              " + str(days_between(javatimeZDT, javatimeZDT.plusDays(5))))
  testdate.log.warn("days_between(-5):              " + str(days_between(javatimeZDT.plusDays(5), javatimeZDT)))
  testdate.log.warn("hours_between(+5):             " + str(hours_between(javatimeZDT, javatimeZDT.plusHours(5))))
  testdate.log.warn("hours_between(-5):             " + str(hours_between(javatimeZDT.plusHours(5), javatimeZDT)))
  testdate.log.warn("minutes_between(+5):           " + str(minutes_between(javatimeZDT, javatimeZDT.plusMinutes(5))))
  testdate.log.warn("minutes_between(-5):           " + str(minutes_between(javatimeZDT.plusMinutes(5), javatimeZDT)))
  testdate.log.warn("seconds_between(+5):           " + str(seconds_between(javatimeZDT, javatimeZDT.plusSeconds(5))))
  testdate.log.warn("seconds_between(-5):           " + str(seconds_between(javatimeZDT.plusSeconds(5), javatimeZDT)))

5iver commented 5 years ago

We should look into what is changing in https://github.com/openhab/openhab-core/pull/945#discussion_r309868785 and get most/all of the functionality in this module moved into OH.

CrazyIvan359 commented 5 years ago

I don't really have time right to learn Java to add this to openhab. Just flipping through that PR on my phone it looks like it is part of an ongoing transition to ZDT?

5iver commented 5 years ago

They are adding helper functions to DateTimeType.

CrazyIvan359 commented 5 years ago

I wonder if most of the content of this module will be considered extraneous. The utilities are all in Java, here we are making things easy but in openHAB they want simplicity right? The converters are also ease of use and not strictly necessary for the core functionality of openhab.

I'm not against integrating the functionality, but is core the right place?

5iver commented 5 years ago

Yes, because that is currently where automation lives. That is likely going to change. If the functionality doesn't go into the classes directly, then it will go into the Scripting API or Actions.

CrazyIvan359 commented 5 years ago

Awesome. We'll of you want to work on it I can consult or maybe help with bits of it. I'm focusing on Eos and my setup right now (it's messy and too many things aren't working). I will do the discussed changes to this module before the end of the weekend though, so we can merge this at least.

5iver commented 5 years ago

This test script...

from core.log import logging, LOG_PREFIX, log_traceback
log = logging.getLogger("{}.TEST".format(LOG_PREFIX))

import personal.date
reload(personal.date)
from personal.date import *

from java.time import ZoneId, LocalDateTime, ZonedDateTime
from java.util import Date
from org.eclipse.smarthome.core.library.types import DateTimeType as eclipseDTT
from org.openhab.core.library.types import DateTimeType as legacyDTT

log.warn("Local timezone")
javatimeZDT = ZonedDateTime.now()
log.warn("javatimeZDT:                                                                                                 {}".format(javatimeZDT))
log.warn("format_date(javatimeZDT):                                                                                    {}".format(format_date(javatimeZDT)))
log.warn("format_date(LocalDateTime.parse(format_date(javatimeZDT, \"yyyy-MM-dd'T'HH:mm:ss\"))):                         {}".format(format_date(LocalDateTime.parse(format_date(javatimeZDT, "yyyy-MM-dd'T'HH:mm:ss")))))
log.warn("to_joda_datetime(javatimeZDT):                                                                               {}".format(format_date(to_joda_datetime(javatimeZDT))))
log.warn("format_date(to_python_datetime(javatimeZDT)):                                                                {}".format(format_date(to_python_datetime(javatimeZDT))))
log.warn("format_date(to_java_calendar(javatimeZDT)):                                                                  {}".format(format_date(to_java_calendar(javatimeZDT))))
log.warn("format_date(Date):                                                                                           {}".format(format_date(Date(javatimeZDT.toInstant().toEpochMilli()))))
log.warn("format_date(eclipseDTT):                                                                                     {}".format(format_date(eclipseDTT(format_date(javatimeZDT)))))
log.warn("format_date(legacyDTT):                                                                                      {}".format(format_date(legacyDTT(format_date(javatimeZDT)))))

log.warn(" ")
log.warn("CST timezone")
javatimeZDT_CST = javatimeZDT.withZoneSameInstant(ZoneId.of("US/Central"))
log.warn("javatimeZDT_CST:                                                                                             {}".format(javatimeZDT_CST))
log.warn("format_date(javatimeZDT_CST):                                                                                {}".format(format_date(javatimeZDT_CST)))
log.warn("format_date(to_joda_datetime(javatimeZDT_CST)):                                                              {}".format(format_date(to_joda_datetime(javatimeZDT_CST))))
log.warn("format_date(to_python_datetime(javatimeZDT_CST)):                                                            {}".format(format_date(to_python_datetime(javatimeZDT_CST))))
log.warn("format_date(to_java_calendar(javatimeZDT_CST)):                                                              {}".format(format_date(to_java_calendar(javatimeZDT_CST))))
log.warn("format_date(Date(javatimeZDT_CST.toInstant().toEpochMilli())) (no timezone info, so local):                  {}".format(format_date(Date(javatimeZDT_CST.toInstant().toEpochMilli()))))
log.warn("format_date(eclipseDTT(format_date(javatimeZDT_CST, \"yyyy-MM-dd'T'HH:mm:ssz\"))):                             {}".format(format_date(eclipseDTT(format_date(javatimeZDT_CST, "yyyy-MM-dd'T'HH:mm:ssz")))))
log.warn("format_date(eclipseDTT(format_date(javatimeZDT_CST))):                                                       {}".format(format_date(eclipseDTT(format_date(javatimeZDT_CST)))))
log.warn("format_date(legacyDTT(format_date(javatimeZDT_CST, \"yyyy-MM-dd'T'HH:mm:ssz\"))) (no timezone info, so local): {}".format(format_date(legacyDTT(format_date(javatimeZDT_CST, "yyyy-MM-dd'T'HH:mm:ssz")))))
log.warn("format_date(legacyDTT(format_date(javatimeZDT_CST))):                                                        {}".format(format_date(legacyDTT(format_date(javatimeZDT_CST)))))

log.warn(" ")
log.warn("Differential methods")
log.warn("days_between(javatimeZDT, javatimeZDT.plusDays(5)):                                                          {}".format(days_between(javatimeZDT, javatimeZDT.plusDays(5))))
log.warn("days_between(javatimeZDT.plusDays(5), javatimeZDT):                                                          {}".format(days_between(javatimeZDT.plusDays(5), javatimeZDT)))
log.warn("hours_between(javatimeZDT, javatimeZDT.plusHours(5)):                                                        {}".format(hours_between(javatimeZDT, javatimeZDT.plusHours(5))))
log.warn("hours_between(javatimeZDT.plusHours(5), javatimeZDT):                                                        {}".format(hours_between(javatimeZDT.plusHours(5), javatimeZDT)))
log.warn("minutes_between(javatimeZDT, javatimeZDT.plusMinutes(5)):                                                    {}".format(minutes_between(javatimeZDT, javatimeZDT.plusMinutes(5))))
log.warn("minutes_between(javatimeZDT.plusMinutes(5), javatimeZDT):                                                    {}".format(minutes_between(javatimeZDT.plusMinutes(5), javatimeZDT)))
log.warn("seconds_between(javatimeZDT, javatimeZDT.plusSeconds(5)):                                                    {}".format(seconds_between(javatimeZDT, javatimeZDT.plusSeconds(5))))
log.warn("seconds_between(javatimeZDT.plusSeconds(5), javatimeZDT):                                                    {}".format(seconds_between(javatimeZDT.plusSeconds(5), javatimeZDT)))

... provided these logs...

2019-08-15 19:48:46.201 [WARN ] [jsr223.jython.TEST] - Local timezone
2019-08-15 19:48:46.202 [WARN ] [jsr223.jython.TEST] - javatimeZDT:                                                                                                 2019-08-15T19:48:46.202-04:00[America/Detroit]
2019-08-15 19:48:46.204 [WARN ] [jsr223.jython.TEST] - format_date(javatimeZDT):                                                                                    2019-08-15T19:48:46.20-0400
2019-08-15 19:48:46.205 [WARN ] [jsr223.jython.TEST] - format_date(LocalDateTime.parse(format_date(javatimeZDT, "yyyy-MM-dd'T'HH:mm:ss"))):                         2019-08-15T19:48:46.00-0400
2019-08-15 19:48:46.207 [WARN ] [jsr223.jython.TEST] - to_joda_datetime(javatimeZDT):                                                                               2019-08-15T19:48:46.20-0400
2019-08-15 19:48:46.209 [WARN ] [jsr223.jython.TEST] - format_date(to_python_datetime(javatimeZDT)):                                                                2019-08-15T19:48:46.20-0400
2019-08-15 19:48:46.210 [WARN ] [jsr223.jython.TEST] - format_date(to_java_calendar(javatimeZDT)):                                                                  2019-08-15T19:48:46.20-0400
2019-08-15 19:48:46.212 [WARN ] [jsr223.jython.TEST] - format_date(Date):                                                                                           2019-08-15T19:48:46.20-0400
2019-08-15 19:48:46.213 [WARN ] [jsr223.jython.TEST] - format_date(eclipseDTT):                                                                                     2019-08-15T19:48:46.20-0400
2019-08-15 19:48:46.215 [WARN ] [jsr223.jython.TEST] - format_date(legacyDTT):                                                                                      2019-08-15T19:48:46.00-0400
2019-08-15 19:48:46.216 [WARN ] [jsr223.jython.TEST] -  
2019-08-15 19:48:46.217 [WARN ] [jsr223.jython.TEST] - CST timezone
2019-08-15 19:48:46.217 [WARN ] [jsr223.jython.TEST] - javatimeZDT_CST:                                                                                             2019-08-15T18:48:46.202-05:00[US/Central]
2019-08-15 19:48:46.219 [WARN ] [jsr223.jython.TEST] - format_date(javatimeZDT_CST):                                                                                2019-08-15T18:48:46.20-0500
2019-08-15 19:48:46.220 [WARN ] [jsr223.jython.TEST] - format_date(to_joda_datetime(javatimeZDT_CST)):                                                              2019-08-15T18:48:46.20-0500
2019-08-15 19:48:46.222 [WARN ] [jsr223.jython.TEST] - format_date(to_python_datetime(javatimeZDT_CST)):                                                            2019-08-15T18:48:46.20-0500
2019-08-15 19:48:46.223 [WARN ] [jsr223.jython.TEST] - format_date(to_java_calendar(javatimeZDT_CST)):                                                              2019-08-15T18:48:46.20-0500
2019-08-15 19:48:46.224 [WARN ] [jsr223.jython.TEST] - format_date(Date(javatimeZDT_CST.toInstant().toEpochMilli())) (no timezone info, so local):                  2019-08-15T19:48:46.20-0400
2019-08-15 19:48:46.226 [WARN ] [jsr223.jython.TEST] - format_date(eclipseDTT(format_date(javatimeZDT_CST, "yyyy-MM-dd'T'HH:mm:ssz"))):                             2019-08-15T18:48:46.00-0500
2019-08-15 19:48:46.233 [WARN ] [jsr223.jython.TEST] - format_date(eclipseDTT(format_date(javatimeZDT_CST))):                                                       2019-08-15T18:48:46.20-0500
2019-08-15 19:48:46.238 [WARN ] [jsr223.jython.TEST] - format_date(legacyDTT(format_date(javatimeZDT_CST, "yyyy-MM-dd'T'HH:mm:ssz"))) (no timezone info, so local): 2019-08-15T19:48:46.00-0400
2019-08-15 19:48:46.239 [WARN ] [jsr223.jython.TEST] - format_date(legacyDTT(format_date(javatimeZDT_CST))):                                                        2019-08-15T18:48:46.00-0400
2019-08-15 19:48:46.243 [WARN ] [jsr223.jython.TEST] -  
2019-08-15 19:48:46.244 [WARN ] [jsr223.jython.TEST] - Differential methods
2019-08-15 19:48:46.249 [WARN ] [jsr223.jython.TEST] - days_between(javatimeZDT, javatimeZDT.plusDays(5)):                                                          5
2019-08-15 19:48:46.250 [WARN ] [jsr223.jython.TEST] - days_between(javatimeZDT.plusDays(5), javatimeZDT):                                                          -5
2019-08-15 19:48:46.254 [WARN ] [jsr223.jython.TEST] - hours_between(javatimeZDT, javatimeZDT.plusHours(5)):                                                        5
2019-08-15 19:48:46.256 [WARN ] [jsr223.jython.TEST] - hours_between(javatimeZDT.plusHours(5), javatimeZDT):                                                        -5
2019-08-15 19:48:46.257 [WARN ] [jsr223.jython.TEST] - minutes_between(javatimeZDT, javatimeZDT.plusMinutes(5)):                                                    5
2019-08-15 19:48:46.261 [WARN ] [jsr223.jython.TEST] - minutes_between(javatimeZDT.plusMinutes(5), javatimeZDT):                                                    -5
2019-08-15 19:48:46.263 [WARN ] [jsr223.jython.TEST] - seconds_between(javatimeZDT, javatimeZDT.plusSeconds(5)):                                                    5
2019-08-15 19:48:46.264 [WARN ] [jsr223.jython.TEST] - seconds_between(javatimeZDT.plusSeconds(5), javatimeZDT):                                                    -5

The ESH (has timezone) and compat1x (does not have timezone) DateTimeTypes confused me for a while, but everything looks good to me!

5iver commented 5 years ago

@CrazyIvan359, could you please resolve the conflicts and I'll merge?

CrazyIvan359 commented 5 years ago

That output looks good. Legacy DateTime uses java.util.Date which does not allow setting the TZ, but does parse it correctly.

I'm not at my desktop right now, what conflicts are there?

5iver commented 5 years ago

Click the Resolve Conflicts button below. You can fix them from the web.

CrazyIvan359 commented 5 years ago

My phone can't handle the editor well enough for me to edit. If you want to do it just accept all incoming changes. It's confused because I changed the order of the functions to match their relevance.

CrazyIvan359 commented 5 years ago

Conflicts resolved!

openhab-scripters / openhab-helper-libraries

Fix date module #193