spotfiresoftware / spotfire-python

Package for Building Python Extensions to Spotfire®
Other
18 stars 6 forks source link

'sbdf.export_data' does not handle timezone-aware 'datetime' objects #30

Closed bbassett-tibco closed 1 year ago

bbassett-tibco commented 2 years ago

Running the following script results in an exception:

import datetime as dt
import pandas as pd
from spotfire import sbdf

t1 = dt.datetime.now()
print("* t1")
print(repr(t1))
print(t1.utcoffset())  # None indicates timezone-unaware
t2 = dt.datetime.now(dt.timezone.utc)
print("* t2")
print(repr(t2))
print(t2.utcoffset())  # timedelta indicates timezone-aware
t3 = dt.datetime.now(dt.timezone(dt.timedelta(hours=-7)))
print("* t3")
print(repr(t3))
print(t3.utcoffset())

x = pd.DataFrame({'t1': [t1], 't2': [t2], 't3': [t3]})
print("* x.dtypes")
print(x.dtypes)

sbdf.export_data(x, "C:/tmp/tz.sbdf")
* t1
datetime.datetime(2022, 9, 28, 19, 54, 50, 670360)
None
* t2
datetime.datetime(2022, 9, 29, 2, 54, 50, 671375, tzinfo=datetime.timezone.utc)
0:00:00
* t3
datetime.datetime(2022, 9, 28, 18, 54, 50, 671375, tzinfo=datetime.timezone(datetime.timedelta(days=-1, seconds=61200)))
-1 day, 17:00:00
* x.dtypes
t1               datetime64[ns]
t2          datetime64[ns, UTC]
t3    datetime64[ns, UTC-07:00]
dtype: object
Traceback (most recent call last):
  File "C:\tmp\tz.py", line 18, in <module>
    sbdf.export_data(x, "C:/tmp/tz.sbdf")
  File "C:\src\spotfire-python\spotfire\sbdf.py", line 169, in export_data
    columns, column_names, column_types, table_metadata, column_metadata = _export_columnize_data(obj,
  File "C:\src\spotfire-python\spotfire\sbdf.py", line 220, in _export_columnize_data
    column_types = {str(k): _ValueTypeId.infer_from_dtype(v, f"column '{str(k)}'") for (k, v) in obj.items()}
  File "C:\src\spotfire-python\spotfire\sbdf.py", line 220, in <dictcomp>
    column_types = {str(k): _ValueTypeId.infer_from_dtype(v, f"column '{str(k)}'") for (k, v) in obj.items()}
  File "C:\src\spotfire-python\spotfire\sbdf.py", line 1082, in infer_from_dtype
    raise SBDFError(f"unknown dtype '{dtype}' in {series_description}")
spotfire.sbdf.SBDFError: unknown dtype 'datetime64[ns, UTC]' in column 't2'

(The t3 column also results in an SBDFError: spotfire.sbdf.SBDFError: unknown dtype 'datetime64[ns, UTC-07:00]' in column 't3'.)

Exporting timezone-aware datetime objects should result in timezone-unaware data being exported (since the SBDF format does not have the capacity to encode timezone information).