Closed copybara-service[bot] closed 3 years ago
[xarray-beam] avoid compatibility checks inside _DiscoverTemplate
The default behavior for compat in xarray.combine_nested() is "no_conflicts", which actually calls the fillna() method on variables in a loop:
out = variables[0] for var in variables[1:]: out = out.fillna(var)
https://github.com/pydata/xarray/blob/55e5b5aaa6d9c27adcf9a7cb1f6ac3bf71c10dea/xarray/core/merge.py#L147-L149
This has quadratic behavior if the variables are stored in dask arrays (the dask graph gets one element larger after each loop iteration).
[xarray-beam] avoid compatibility checks inside _DiscoverTemplate
The default behavior for compat in xarray.combine_nested() is "no_conflicts", which actually calls the fillna() method on variables in a loop:
out = variables[0] for var in variables[1:]: out = out.fillna(var)
https://github.com/pydata/xarray/blob/55e5b5aaa6d9c27adcf9a7cb1f6ac3bf71c10dea/xarray/core/merge.py#L147-L149
This has quadratic behavior if the variables are stored in dask arrays (the dask graph gets one element larger after each loop iteration).