Support Timestamp Generator Type - Githubissues

deephaven / benchmark

Deephaven Benchmarking

Apache License 2.0

0 stars 3 forks source link

Support Timestamp Generator Type #39

Closed stanbrub closed 1 year ago

stanbrub commented 1 year ago

Deephaven query operations like Time-based EMA and Time-based rolling sum need a timestamp column to function.

How much control over the timestamp is needed from the generator side?
- Are out of order timestamps required?
- Millisecond precision good enough?
- This doesn't fit with the notion of random data
This would have to work with generateParquet(), generateAvro(), and possibly generateSQL()

Could be done like the following for now:

Another column type: <.add("timeCol", "timestamp", "[now]")>
A generator directive: <.withTimestamp("timeCol")> that would grab the current time on publish

stanbrub commented 1 year ago

Added timestamp-millis to data generator and to avro schema generator. It uses millis since epoch range (ex. [1-100]) for now. So far, the time-window tests don't need actually times or even timezones to function.