search

doublep / datetime

Library for parsing, formatting, matching and recoding timestamps and date-time format strings.
GNU General Public License v3.0
19 stars 4 forks source link

readme

:toc: macro :toc-title: Table of contents :source-language: lisp ifndef::env-github[:icons: font] ifdef::env-github[] :warning-caption: :warning: :caution-caption: :fire: :important-caption: :exclamation: :note-caption: :paperclip: :tip-caption: :bulb: endif::[] :uri-icu: https://unicode-org.github.io/icu/userguide/datetime/ :uri-java-datetimeformatter: https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/time/format/DateTimeFormatter.html :uri-php-date: http://php.net/manual/en/function.date.php :uri-logview: https://github.com/doublep/logview

image:https://img.shields.io/badge/license-GPL_3-green.svg[License: GPL 3, link=http://www.gnu.org/licenses/gpl-3.0.txt] image:https://img.shields.io/github/release/doublep/datetime.svg[Latest release, link=https://github.com/doublep/datetime/releases] image:http://stable.melpa.org/packages/datetime-badge.svg[MELPA Stable, link=http://stable.melpa.org/#/datetime] image:https://github.com/doublep/datetime/workflows/CI/badge.svg[CI, link=https://github.com/doublep/datetime/actions?query=workflow%3ACI]

= datetime

Datetime is a library for parsing, formatting, matching and recoding timestamps and date-time format strings. Not all of the planned functionality is implemented yet.

== System locale and timezone

The library will try to guess your system locale and timezone, but this is frustratingly difficult. In particular, on MS Windows it will not be able to determine the timezone (not sure about locale). Patches to improve this are welcome.

In any case, when it fails completely or guesses incorrectly, you can always override the heuristics results by setting variables datetime-locale and/or datetime-timezone manually. Both are also available through Emacs customization interface, group datetime.

To find the list of all supported locales and timezones, evaluate the following forms:

.... (prin1-to-string (sort (datetime-list-locales t) #'string<)) (prin1-to-string (sort (datetime-list-timezones) #'string<)) ....

== Overall goals

This library is targeted at bulk-processing, therefore many functions are optimized for speed, but not necessarily for ease of use. For example, <<#formatting,formatting>> is done in two steps: first you need to generate a formatting function for given pattern, and only using it obtain formatted strings.

The initial and primary user of the library is {uri-logview}[Logview], where it is used to quickly <<#matching,match>> thousands of timestamps (usually one per line) in a log file, so speed was very important from the start. Later it also started using datetime’s <<#parsing,parsing>> functionality in a similar way — meaning that speed was important there too.

== Pattern types

There exist several different ways to specify date and/or time format. Different programming languages and their libraries use different formats. E.g. format of date 2015-09-24 can be expressed at least in the following ways:

This library currently uses Java pattern format everywhere, but is internally written in such a way that support for other types can be added relatively easily — when someone needs them.

=== Supported Java pattern elements

Java {uri-java-datetimeformatter}[date-time patterns] are very versatile and support a lot of features. This library doesn’t implement — at least currently — every element, but does support all the most imporant ones. In particular, everything (with the exception of timezone names) needed by patterns returned from the three datetime-locale-*-pattern functions is supported in all other parts of the library. Timezone names are currently supported when <<#formatting,formatting>> only.

Here is an overview of Java date-time pattern elements together with their support in datetime library:

[%autowidth] |=== ^| Symbol | Meaning ^| Support in the library

^| G | era (AD/BC) ^| full ^| u | year ^| none ^| y | year of era ^| full ^| Y | week-based year ^| full ^| Q/q | quarter of year ^| none ^| M/L | month of year ^| full ^| w | week of week-based year ^| full ^| W | week of month ^| full ^| D | day of year ^| full ^| d | day of month ^| full ^| g | modified julian day ^| none ^| E | day of week ^| full ^| e/c | localized day of week ^| full ^| F | day-of-week in month ^| full ^| a | AM/PM ^| full ^| B | period of day ^| full ^| H | hour of day (0-23) ^| full ^| k | hour of day (1-24) ^| full ^| h | hour in AM/PM (1-12) ^| full ^| K | hour in AM/PM (0-11) ^| full ^| m | minute of hour ^| full ^| s | second of minute ^| full ^| S | fraction of second ^| full ^| A | millisecond of day ^| none ^| n | nanosecond of second ^| none ^| V | timezone id ^| none ^| v | generic timezone name ^| none ^| z | timezone name ^| only formatting ^| Z | timezone z-offset ^| full ^| x | timezone x-offset ^| full ^| X | timezone x-offset or 'Z' for zero ^| full ^| O | localized timezone offset ^| full ^| p | pad modifier ^| none ^| ' | quoting for literal text ^| full ^| [/] | optional section ^| none |===

Some examples of commonly used patterns:

[%autowidth] |=== | Pattern | Example timestamp | Notes

| yyyy-MM-dd HH:mm:ss | 2023-09-18 21:29:02 | | yyyy-MM-dd HH:mm:ss.SSS | 2023-09-18 21:29:02.618 | | yyyy-MM-dd\'T\'HH:mm:ssZ | 2023-09-18T21:29:02+0200 | | yyyy-MM-dd HH:mm:ssXXX | 2023-09-18 21:29:02+02:00 | | yyyy-MM-dd HH:mm:ssx | 2023-09-18 21:29:02+02 | | yyyy-MM-dd HH:mm:ss O | 2023-09-18 21:29:02 GMT+2 | | yyyy-MM-dd HH:mm:ss z | 2023-09-18 21:29:02 CEST | only formatting supported | EEE MMM dd HH:mm:ss yyyy | Mon Sep 18 21:29:02 2023 | | dd.MM.yyyy | 18.09.2023 | | MMM d, yyyy | Sep 18, 2023 | | h:mm:ss a | 9:29:02 PM | |===

[#parsing] == Parsing timestamps

Parsing timestamps using the library is a two-step operation. First you create a parser function for a specific pattern and options. Then call the parser function with one argument — the timestamp as a string. It returns a floating point number — number of seconds since UNIX epoch in UTC timezone.

Create a parser:

.... (datetime-parser-to-float 'java "yyyy-MM-dd HH:mm:ss.SSS" :timezone 'system) ....

And use it:

.... (let ((parser (datetime-parser-to-float 'java "yyyy-MM-dd HH:mm:ss.SSS" :timezone 'system))) (funcall parser "2018-09-18 21:20:00.000")) ....

Remember that if you parse timestamps formatted on your machine, you need to pass 'system as :timezone option to datetime-parser-to-float: default timezone is UTC.

Parsing timestamps with varying timezones (i.e. with timezone information directly in the input string) has limited support as of 0.9: you can now parse timezone offsets, but not yet timezone names. E.g. “20:00:00+01” is parseable (for example, with pattern “HH:mm:ssx"), but “20:00:00 CET” cannot really be parsed.

[#formatting] == Formatting timestamps

To format timestamps you first need to create a formatter function. This function accepts one argument — the timestamp as a floating point number — and converts it to a string. All customization, most importantly, specifying the pattern, is done at the point of creating the formatter.

For example:

.... (datetime-float-formatter 'java "yyyy-MM-dd HH:mm:ss.SSS" :timezone 'system) ....

With this formatter function you can now format timestamps as follows:

.... (let ((formatter (datetime-float-formatter 'java "yyyy-MM-dd HH:mm:ss.SSS" :timezone 'system))) (funcall formatter (float-time))) ....

Note that if you plan to format result of float-time function, you need to pass 'system as :timezone option to datetime-float-formatter: default timezone is UTC.

As of version 0.9 the library fully supports formatting timezones: both names (z and zzzz in Java patterns) and offsets (z, x, X, O; in various repetition counts) can be used to format abbreviated of full names and offsets to GMT. For example:

.... (let ((formatter1 (datetime-float-formatter 'java "HH:mm:ss z" :timezone 'system)) (formatter2 (datetime-float-formatter 'java "HH:mm:ssx" :timezone 'system))) (cons (funcall formatter1 (float-time)) (funcall formatter2 (float-time)))) ....

[#matching] == Matching timestamps

Sometimes you need to determine if given string is (likely) a timestamp, corresponding to given pattern. A robust way, of course, is to try to parse it. However, it is much faster, though not as precise, to use a regular expression.

Function datetime-matching-regexp builds such a regular expression for given pattern. For example,

(datetime-matching-regexp 'java "yyyy-MM-dd HH:mm:ss.SSS")

returns a regexp that will match all timestamp strings produced by the formatter we created earlier. It will also match some other strings, but is good enough in practice to tell if “this does look like a timestamp”.

Timezone support in matching is currently the same as for <<#parsing,parsing>>.

== Other functions

These functions are also part of the library interface. They are documented within Emacs.