Open riley-brady opened 1 year ago
Another side note that came up here -- I'm curious if there's any roadmap plan for recognizing integration of units for methods like rolling().sum()
.
E.g.,
data = xr.DataArray(range(3), dims='time').pint.quantify('mm/day')
data.pint.units
>>> mm/day
data = data.rolling(time=2).sum()
data.pint.units
>>> mm
thanks for the report, @riley-brady. It seems that xarray
operations on pint
+dask
are not as thoroughly tested as pint
and dask
on their own. I think this is a bug in pint
(or dask
, not sure): we enable force_ndarray_like
to convert scalars to 0d arrays, which means that the final call to np.pad
becomes:
np.pad(magnitude, pad_width, mode="constant", constant_values=np.array(0))
numpy
seems to be fine with that, but dask
is not.
@jrbourbeau, what do you think? Would it make sense to extend expand_pad_value
to unpack 0d arrays (using .item()
), or would you rather have the caller (pint
, in this case) do that?
I'm curious if there's any roadmap plan for recognizing integration of units for methods like
rolling().sum()
I'm not sure I follow. Why would rolling().sum()
work similar to integration, when all it does is compute a grouped sum? I'm not sure if this actually counts as integration, but you can multiply the result of the rolling sum with the diff
of the time coordinate (which is a bit tricky because time
is an indexed coordinate):
data = xr.DataArray(
range(3), dims="time", coords={"time2": ("time", [1, 2, 3])}
).pint.quantify("mm/day", time="day")
dt = data.time2.pad(time=(1, 0)).diff(dim="time")
data.rolling(time=2).sum() * dt
and then you would have the correct units (with the same numerical result, because I chose the time coordinate to have increments of 1 day
)
Thanks for the quick feedback on this issue @keewis.
Also thanks for the demo with .diff()
. You're right about the integration assumptions. In my specific use case I am doing a rolling sum of units mm/day
with daily time steps, so in this case it should reflect total precip in mm
, but that's not a fair assumption for many other cases. I'll give the .diff()
method a try.
this should be fixed in dask
since quite a while ago, but I'll leave it open until we have tests for this (probably after copying the test suite from xarray
)
Hi folks,
I noticed that when running
.rolling(...)
on a chunkedpint
array, there is an exception raised that breaks the process:I outline three different cases below for running
.rolling()
on apint
-awareDataArray
.pint
array.xr.set_options(use_bottleneck=False)
preserves units (https://github.com/pydata/xarray/issues/7062#issuecomment-1254047656).pint
array, usingxarray
chunking.bottleneck
. However, this isn't an optimal solution for me, since one cannot queryds.pint.units
on anxarray
-chunkedpint
array. I like being able to do that for various QOL checks in a data pipeline.pint
array chunked withds.pint.chunk(...)
.bottleneck
.My solution in the interim is to do something like: