DataONEorg / metrics-service

An efficient database and REST API for delivering aggregated data set metrics to clients.
Apache License 2.0
2 stars 1 forks source link

Make date parsing more flexible #96

Open datadavev opened 9 months ago

datadavev commented 9 months ago

https://github.com/DataONEorg/metrics-service/blob/696b1247dd0cb5487336cf8e1995f373d83f0d37/src/d1_metrics/d1_metrics/es_sysmeta_sync.py#L544

Seeing exception in /var/log/dataone/identifier_es/application.log:

Traceback (most recent call last):
  File "/home/vieglais/.local/bin/d1sysmetaes", line 11, in <module>
    load_entry_point('d1-metrics', 'console_scripts', 'd1sysmetaes')()
  File "/home/vieglais/git/metrics-service/src/d1_metrics/d1_metrics/es_sysmeta_sync.py", line 666, in main
    start_date = findMostRecentRecord(record_file_path)
  File "/home/vieglais/git/metrics-service/src/d1_metrics/d1_metrics/es_sysmeta_sync.py", line 544, in findMostRecentRecord
    tstamp = datetime.datetime.strptime(tstamp, "%Y-%m-%dT%H:%M:%S.%fZ")
  File "/usr/lib/python3.6/_strptime.py", line 565, in _strptime_datetime
    tt, fraction = _strptime(data_string, format)
  File "/usr/lib/python3.6/_strptime.py", line 362, in _strptime
    (data_string, format))

A simple fix is to catch the error and try a fallback time pattern.

datadavev commented 9 months ago

e.g.:

        try:
            tstamp = datetime.datetime.strptime(tstamp, "%Y-%m-%dT%H:%M:%S.%fZ")
        except Exception as e:
            tstamp = datetime.datetime.strptime(tstamp, "%Y-%m-%dT%H:%M:%SZ")