Fix time zone handling in index values and metadata - Githubissues

man-group / ArcticDB

ArcticDB is a high performance, serverless DataFrame database built for the Python Data Science ecosystem.

http://arcticdb.io

Other

1.52k stars 93 forks source link

Fix time zone handling in index values and metadata #1866

Open G-D-Petrov opened 1 month ago

G-D-Petrov commented 1 month ago

Reference Issues/PRs

Fixes #1406 and #1827

What does this implement or fix?

Unify the way time zones are handled for indexes and metadata, as much as possible
Extend the support for time zone in the indexes/metadata to cover all cases supported by pandas
Update the documentation to be more explicit about how/where time zones are supported

This is done by extending the current approach to decompose the timestamp into:

its value - int
its time zone info - str

This approach is beneficial as it:

keeps the interface the same
yields the most consistent results

The main problem is that we need to do some post processing to coerce the time zone info to be consistent (done here)

Any other comments?

An alternative approach was considered in a past PR here, but that had a couple of limitation:

required an interface break
was inconsistent because:
- getting the offset value in some cases was incorrect inputs of type datetime
- still needed to do some post processing to make sure that the time zone info is in a consistent format