JuliaDataReaders / DataReaders.jl

A Julia library to get remote data via Requests.jl and get DataFrame (from DataFrames.jl) or TimeArray (from TimeSeries.jl)
MIT License
10 stars 7 forks source link

Flexible date parsing #13

Open femtotrader opened 8 years ago

femtotrader commented 8 years ago

Should we use flexible date parsing ?

see http://comments.gmane.org/gmane.comp.lang.julia.user/36468

julia> length(long_list_of_date_strs)
497338

julia> function find_matching_datetime_format(datestr)
           datestr = strip(datestr)
           formats = ["y-m-d H:M", "y/m/d H:M", "y m d H:M"] # Add more formats as needed...
           for ft in formats
               try
                   d = Dates.DateTime(datestr, ft)
                   return d, ft
               catch err
               end
           end
           throw("Cannot find a date format that matches: $(datestr)")
       end
find_matching_datetime_format (generic function with 1 method)

julia> parsedate(s) = find_matching_datetime_format(s)[1]
parsedate (generic function with 1 method)

julia> function parsedates(datestrings)
           d, ftstr = find_matching_datetime_format(datestrings[1])
           ft = Dates.DateFormat(ftstr)
           map(ds -> Dates.DateTime(ds, ft), datestrings)
       end
parsedates (generic function with 1 method)

julia> <at> time r1 = map(parsedate, long_list_of_date_strs[1:10000]);
elapsed time: 3.507318034 seconds (273707960 bytes allocated, 41.44% gc time)

julia> <at> time r2 = parsedates(long_list_of_date_strs[1:10000]);
elapsed time: 0.325861155 seconds (38314796 bytes allocated, 72.21% gc time)

julia> r1 == r2
true

For millisecond resolution

julia> DateTime("20151125 04:07:35.919", "yyyymmdd HH:MM:SS.sss")
2015-11-25T04:07:35.919