The PR removes some panics that could happen in datetime conversions. In particular, we were using arrow's unary function to map a custom function across the elements of an arrow array. For performance reasons, the function will be run on the underlying arrow memory even when the corresponding element is null, so the function must be valid for any possible value of the type. For the Date/timestamp types, this causes problems because not all valid integers represent value Dates or Timestamps. This PR switches to using the try_unary function which only evaluates the input function on non-null elements. In addition, I removed a bunch of unwrap with proper error propagation.
The PR removes some panics that could happen in datetime conversions. In particular, we were using arrow's
unary
function to map a custom function across the elements of an arrow array. For performance reasons, the function will be run on the underlying arrow memory even when the corresponding element isnull
, so the function must be valid for any possible value of the type. For the Date/timestamp types, this causes problems because not all valid integers represent value Dates or Timestamps. This PR switches to using thetry_unary
function which only evaluates the input function on non-null elements. In addition, I removed a bunch ofunwrap
with proper error propagation.