Support freezegun-manipulated datetimes

[lorenzwalthert]

Problem description

In some of our unit tests and benchmarks, we use the relatively popular freezegun package to manipulate system time. However, when I try to create a data frame that consists such a mocked date time, I get an error:

>>> import datetime
>>> import polars
>>> from freezegun import freeze_time

>>> with freeze_time("2012-01-14"):
...       polars.DataFrame({'dt': datetime.datetime.now()})

Traceback (most recent call last):
File "<stdin>", line 2, in <module>
File "/Users/lorenz/git/epex-spot-intraday/.venv/lib/python3.10/site-packages/polars/utils/decorators.py", line 35, in wrapper
   return function(*args, **kwargs)
File "/Users/lorenz/git/epex-spot-intraday/.venv/lib/python3.10/site-packages/polars/dataframe/frame.py", line 357, in __init__
   self._df = dict_to_pydf(
File "/Users/lorenz/git/epex-spot-intraday/.venv/lib/python3.10/site-packages/polars/internals/construction.py", line 694, in dict_to_pydf
   for s in _expand_dict_scalars(
File "/Users/lorenz/git/epex-spot-intraday/.venv/lib/python3.10/site-packages/polars/internals/construction.py", line 623, in _expand_dict_scalars
   updated_data[name] = pli.Series(
File "/Users/lorenz/git/epex-spot-intraday/.venv/lib/python3.10/site-packages/polars/series/series.py", line 261, in __init__
   self._s = sequence_to_pyseries(
File "/Users/lorenz/git/epex-spot-intraday/.venv/lib/python3.10/site-packages/polars/internals/construction.py", line 332, in sequence_to_pyseries
   py_series = PySeries.new_from_anyvalues(name, values, strict)
RuntimeError: BindingsError: "object type not supported FakeDatetime(2012, 1, 14, 0, 0)"

Solution

It would be great if bindings for this object type would be supported. We are transitioning from pandas and there it seemed to always have worked out of the box ~(could not find an issue in GitHub with 'freezegun' in it)~. Thanks a lot for this library, it's a game changer for us. Hear my system info FWIW.

>>> polars.show_versions()
---Version info---
Polars: 0.16.14
Index type: UInt32
Platform: macOS-13.2.1-arm64-arm-64bit
Python: 3.10.6 (v3.10.6:9c7b4bd164, Aug  1 2022, 17:13:48) [Clang 13.0.0 (clang-1300.0.29.30)]
---Optional dependencies---
numpy: 1.24.2
pandas: 1.5.3
pyarrow: <not installed>
connectorx: <not installed>
deltalake: <not installed>
fsspec: <not installed>
matplotlib: <not installed>
xlsx2csv: <not installed>
xlsxwriter: <not installed>

[alexander-beedie]

Ref: https://github.com/pola-rs/polars/issues/6713.

[MarcoGorelli]

I'll give this a go after work, I think it should be possible

[lorenzwalthert]

Ref: https://github.com/pola-rs/polars/issues/6713.

Ups, thanks @alexander-beedie, I misspelled freezegun in my search. Updated issue title and comment.

[MarcoGorelli]

@nezinomas just FYI, this should work in the next release

[aberres]

@MarcoGorelli @ritchie46 Is it possible that this partially broke with the 1.0 release?

I am right now upgrading from 0.20.31 to 1.6 and now the following code fails:

import datetime
import polars as pl

from freezegun import freeze_time

with freeze_time("2012-01-14"):
    pl.lit(datetime.date.today())

Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
  File "/Users/armin/venv/weplan3.11/lib/python3.11/site-packages/polars/functions/lit.py", line 132, in lit
    return wrap_expr(plr.lit(value, allow_object=False))
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: cannot create expression literal for value of type FakeDate: FakeDate(2012, 1, 14)

Hint: Pass `allow_object=True` to accept any value and create a literal of type Object.

[MarcoGorelli]

thanks @aberres for reporting - it should work in 1.5.x, it regressed in 1.6.0

https://github.com/pola-rs/polars/pull/18497 should address it, will get back to it soon-ish (and if I can't fix it in time, i'd suggest we just revert the PR that introduced the regression, no big deal)

[aberres]

@MarcoGorelli Great, thanks for looking into this.

I will upgrade to 1.5 then and skip 1.6.