closeio / ciso8601

Fast ISO8601 date time parser for Python written in C

MIT License

565 stars 45 forks source link

Backport datetime.timezone to older Python versions? #45

Closed movermeyer closed 3 years ago

movermeyer commented 6 years ago

Starting in Python 3.7, datetime.timezone objects became accessible through the C-API.

ciso8601 v2.0.0 is scheduled to take advantage of this when compiled against Python 3.7+. The performance improvements are dramatic.

ciso8601 should consider creating a C implementation of a tzinfo subclass. This could be a backport of cPython's timezone class, or a new implementation. Having such an implementation would:

allow Python < 3.7 to experience the same performance gains
allow ciso8601 to sever its dependency on pytz.

movermeyer commented 6 years ago

Note that this would give the largest benefits to Python 2.7 users (~3x speedup over using pytz).

Python [3.2, 3.6] can use the Python interface to get access to datetime.timezone, which is not quite as fast as using the C interface, but still gives nice performance improvements over using pytz. While these versions could still benefit, the amount would not be as much (30% faster over using the Python interface).

Still, if the goal is to be the fastest possible ISO 8601 parser in Python, this is a pretty interesting area of improvement.

movermeyer commented 6 years ago

Note: we could probably copy Pendulum's implementation.

movermeyer commented 6 years ago

I've done an experiment where I integrated Pendulum's timezone implementation into ciso8601 v2.0.1.

The speedups are nearly as dramatic as I estimated above:

Python 2.7

flame_graph_tzinfo_27

That's almost exactly 3 times faster!

(Aside: this flame graph doesn't look as pretty as the others (because of Python 2,7), and is missing pendulum (due to a regression))

Python 3.6

flame_graph_tzinfo_36

54% faster

Python 3.7

Even Python 3.7 benefits

flame_graph_tzinfo_37

24% faster

Next Steps

[x] Integrate the pendulum tzinfo code, including their (MIT) license.
- [x] Make it work on Windows
[ ] Figure out a nice way to give acknowledgements to Pendulum's author.

movermeyer commented 5 years ago

Before we can do this, we should confirm that we are able to pickle objects that use pendulum's FixedOffset timezones. There was a bug related to this raised in another project.

👍 Fixed as part of https://github.com/closeio/ciso8601/pull/58 and https://github.com/movermeyer/backports.datetime_fromisoformat/pull/13