search

quickwit-oss / quickwit

Cloud-native search engine for observability. An open-source alternative to Datadog, Elasticsearch, Loki, and Tempo.
https://quickwit.io
Other
7.43k stars 301 forks source link

Support specifying a timezone manually with strptime parser #3171

Open guilload opened 1 year ago

guilload commented 1 year ago

For instance:

name: timestamp
type: datetime
description: Time at which the event was emitted
input_formats:
  - rfc3339
  - unix_timestamp
  - "%Y %m %d %H:%M:%S.%f %z"
  - strptime:
      format: "%Y %m %d %H:%M:%S.%f"
      default_timezone: America/New_York
fulmicoton commented 1 year ago

Maybe default_timezone?

guilload commented 1 year ago

default_timezone sounds better, indeed.

rdettai commented 6 days ago

I think we should raise an error during the parsing of the config if the user provides a format with a timezone and a default timezone (%z or %Z). It's better to fail early. Any opinion @fulmicoton ?

rdettai commented 6 days ago

Parsing a timezone like America/New_York and use it to compute a timestamp from a date string requires a TZ database. The most popular / better maintained one seems to be chrono-tz, but we use time almost everywhere in Quickwit (and Tantivy). There is a time-tz crate that would fit more nicely into the existing code, but it appears to be substantially less used.