search

apache / arrow

Apache Arrow is the universal columnar format and multi-language toolbox for fast data interchange and in-memory analytics
https://arrow.apache.org/
Apache License 2.0
14.65k stars 3.55k forks source link

[FlightRPC][C++] Precision of `arrow.flight.Timestamp` is OS-dependent #44679

Closed EnricoMi closed 1 week ago

EnricoMi commented 2 weeks ago

Describe the bug, including details regarding any error messages, version, and platform.

The precision of is supposed to be nanoseconds precision: https://github.com/apache/arrow/blob/093655c60783321c786a8e69632f185d37520f4d/cpp/src/arrow/flight/types.h#L77

Actually, the current precision of arrow.flight.Timestamp not nanoseconds on most operation systems: https://github.com/apache/arrow/blob/093655c60783321c786a8e69632f185d37520f4d/cpp/src/arrow/flight/types.h#L83 The precision of std::chrono::system_clock::time_point is

https://github.com/apache/arrow/blob/093655c60783321c786a8e69632f185d37520f4d/cpp/src/arrow/flight/flight_internals_test.cc#L187-L192

In order to be OS-agnostic, the timestamp resolution should be nanoseconds everywhere.

In Python, the equivalent type TimePoint is correctly defined with nanoseconds precision for any OS: https://github.com/apache/arrow/blob/093655c60783321c786a8e69632f185d37520f4d/python/pyarrow/src/arrow/python/datetime.h#L88-L89

Component(s)

C++

lidavidm commented 1 week ago

Issue resolved by pull request 44681 https://github.com/apache/arrow/pull/44681