Closed jeffreyssmith2nd closed 1 month ago
Hi @jeffreyssmith2nd,
Thank you for testing our implementation and for your detailed feedback.
Let me clarify the intended use and behavior of the queryPoints
and query
functions within our system:
query(): This function is designed to return a stream of rows, with each row mapped as a friendly type object from the vector, as defined by Arrow's ValueVector.getObject(int). The purpose of query
is to serialize data to values depending on their Arrow Flight Type. If the underlying Flight type changes, the returned values may also change accordingly. For example, the formatted string 2020-06-11T16:54:30
represents the toString value of the original Arrow Flight Type probably DateMilliVector
.
queryPoints(): This function returns a stream of points (com.influxdb.v3.client.PointValues
), which are structured for potential re-ingestion into the database after operations like downsampling. The PointValues
are structured to store values correctly for database ingestion, e.g., timestamps in nanoseconds.
Regarding your observations and the discrepancy when timezones are enabled:
1591894470000000000000000
from queryPoints
was indeed a bug, which we have addressed in fix #153. The correct behavior should reflect nanosecond precision appropriate for database operations.1591894470000000000
result from query
reflects the raw nanosecond timestamp as it should appear without any pretty formatting, which aligns with the Arrow Flight Type's representation.For pretty printing:
PointValue.toString
method to include human-readable formatting, like 2020-06-11T16:54:30.123456789
, is something we have to do for future updates. This enhancement would align the output more closely with user expectations for readability.Thank you once again for your engagement! 👍
Best
The results were parsed as follows:
TimeStampNanoVector
mapped to LocalDateTime
.TimeStampNanoTZVector
mapped to Long
.Using our fixed version of the client (#153) in conjunction with queryPoints
will ensure that the results are always correct timestamps in nanoseconds.
Specifications
main
branchmain
as of July 12thCode sample to reproduce problem
Expected behavior
Whether timezone support is enabled or not, I would expect all times to be treated with Nanosecond precision. When printing the raw number format, I would expect a number like
1591894470000000000000000
. When pretty printing the time, I would expect a string like2020-06-11T16:54:30.123456789
(where the digits after the decimal could be omitted if they were all zeros).If timezone support is enabled, I would expect the pretty printed format to also have a
Z
at the end. The Flight Python libraries are a good reference for the behaviour I would expect.Actual behavior
We recently (Mon July 8th) enabled timezone support in one serverless environment (us-east-1) and have noticed unexpected behaviour in the Java client. My test code executes a
queryPoints
andquery
, using the same SQL.If I run without timezones enabled (the default behaviour before Monday) I get the following results:
If I run with timezones enabled (the behaviour now) I get the following results:
Note that in both cases (timezone enabled/disabled), the times are always returned in Nanoseconds but the Java client does not always display them in Nanoseconds.
Additional info
No response