Closed amirouche closed 2 years ago
Good point. In fact, the date format detection could be implemented in a way similar to that of type detection. I.e., test dates record by record until a given confidence threshold has been crossed.
To clarify, process.type_cast
calls convert.to_date
and convert.to_datetime
, and those functions then eventually call dateutil.parser.parse
. parse
takes an optional dayfirst
parameter which defaults to False
. This should be configurable, and optionally detected by a new function.
:100:
Right now, the type detection does infer a date, datetime or time types without taking into account the fact that
01/02/2002
can be both Februrary the 1st or January the 2nd depending on the date format used respectivelyDD/MM/YYYY
andMM/DD/YYYY
.This might be undecidable in some rare cases, but in general it's possible given enough values to decide between both formats.
One possible way, to handle this in
meza
is to use a higher level datatype for representing the type of a field to replace the current string representation. For instance:Basically, use a representation that takes optional extra information about the type.