dalibo / pgtoolkit

Postgres Support from Python
https://pgtoolkit.rtfd.io/
PostgreSQL License
21 stars 8 forks source link

Relax date format requierment (UTC) #91

Open blogh opened 2 years ago

blogh commented 2 years ago

Hi,

https://github.com/dalibo/pgtoolkit/blob/249034a214cddc223b7b7d4162f056ceb93899ca/pgtoolkit/log/parser.py#L128

Would it be possible to have an option to bypass this requirement ? I wanted to use this class to handle log parsing in a project where I just needed the date "as is" and couldn't because of this.

dlax commented 2 years ago

Looks reasonable. By the way, it's not clear to me why datetime.strptime() is not used directly here.

blogh commented 2 years ago

By the way, it's not clear to me why datetime.strptime() is not used directly here.

maybe it's because there are several formats in log_line_prefix ?

https://www.postgresql.org/docs/14/runtime-config-logging.html#GUC-LOG-LINE-PREFIX

  # %m (Time stamp with milliseconds)+ log_timezone = Europe/Paris
>>d = datetime.datetime.strptime("2021-09-22 19:00:45.894 CET", "%Y-%m-%d %H:%M:%S.%f %Z")

  # %t (Time stamp without milliseconds)
>>d = datetime.datetime.strptime("2021-12-07 10:14:04 CET", "%Y-%m-%d %H:%M:%S %Z")

--# %n (Time stamp with milliseconds (as a Unix epoch)) (not supported in pgtoolkit, never seen it, do we want it ?)
>>d = datetime.datetime.fromtimestamp(1638868691.099)
dlax commented 2 years ago

We could try different parsing methods (strptime, or even datetime.fromisoformat() when there is no tz info). Another option would be to use https://dateutil.readthedocs.io/en/stable/ but that would pull a dependency (the first, IIRC).

@blogh, if you need something to be changed in some way, feel free to submit a patch.