search

twosigma / flint

A Time Series Library for Apache Spark
Apache License 2.0
993 stars 184 forks source link

Failure of test: "correctly convert SQL TimestampType with default format" in "CSVSpec.scala". #39

Open alonso-santaella opened 5 years ago

alonso-santaella commented 5 years ago

Running sbt assembly yields the following:

[info] - should correctly convert SQL TimestampType with default format *** FAILED *** (244 milliseconds)
[info]   1199232000000000000 did not equal 1199253600000000000 (CSVSpec.scala:100)

Now, 1199232000000000000, the result of first.getAs[Long]("time"), where first contains the string "2018-01-02 00:00:00" equals 2018-01-01 18:00:00, which is in turn equal to "2018-01-02 00:00:00 UTC-6."

1199253600000000000, on the other hand, equals "2018-01-02 00:00:00" and since its timezone was extracted from format.setTimeZone(TimeZone.getDefault) (my computer resides in Mexico City, UTC-6), I believe it is being interpreted as "2018-01-02 00:00:00 UTC-6".

tl;dr: I believe the test is reading "2018-01-02 00:00:00" from CSV and assuming UTC and then converting it to UTC-6, yet simply appending my default timezone to the hard-coded string "2018-01-02 00:00:00".