teragrep / pth_10

Data Processing Language (DPL) translator for Apache Spark
GNU Affero General Public License v3.0
0 stars 6 forks source link

Use Instant for timestamps in Spark 3 version #209

Open eemhu opened 7 months ago

eemhu commented 7 months ago

Description Use Instant to allow for the widest earliest-latest range since it is supported on Spark 3.

Use case or motivation behind the feature request Allows direct usage of Instant instead of worrying about long overflows that can be caused by using Instant and converting it to epoch.

Related issues

Additional context https://issues.apache.org/jira/browse/SPARK-26902