search

aws-samples / lookout-for-equipment-demo

MIT No Attribution
25 stars 27 forks source link

A problem with scheduler's local time zone adjustment #2

Open youngmki opened 3 years ago

youngmki commented 3 years ago

Hello. I'm working as a Data Scientist at AWS ProServe Korea, and I've tested Lookout for Equipment to try it out on an enterprise project. Then I found a point in need of improvement.

When the scheduler finds the timestamp in the data file name for the current time, I can adjust the local time zone by specifying the 'INPUT_TIMEZONE_OFFSET' parameter. For example, if the current is April 3rd at 12:00 AM UTC, in Korea (UTC + 9) I can enter the parameter as '+9:00' and specify the timestamp of the file name as '20210403090000'. This works fine.

However, when the scheduler filters the timestamp column inside the data file, it does not take into account local time zone adjustments. In other words, if the timestamp field is '2021-04-03 09:00:00', I get an error. It should be put in UTC standard.

This difference is confusing to customers and inconvenient because you have to deal with file names and data timestamps in different ways. I hope my feedback is helpful. Thank you.

michaelhoarau commented 3 years ago

Hi, Thanks, I've put together a short example to illustrate this discrepancy for the team. Thanks for the suggestion to improve the customer experience with this feature! Michaël.