This changes the Xarray-Beam data model to keep track of chunk keys
in a different way: as a pair of (offsets, variables), rather than only
offsets. The new key type is given a different name (Key rather than ChunkKey)
because it also has a completely different API.
This should allow for more efficient pipelines, e.g., by splitting datasets
across variables for rechunking.
It also includes the new transforms SplitVariables() and ConsolidateVariables(),
and exposes a number of internal helper functions as public APIs.
[xarray-beam] breaking refactor: replace ChunkKey with Key
Fixes https://github.com/google/xarray-beam/issues/9
This changes the Xarray-Beam data model to keep track of chunk keys in a different way: as a pair of (offsets, variables), rather than only offsets. The new key type is given a different name (Key rather than ChunkKey) because it also has a completely different API.
This should allow for more efficient pipelines, e.g., by splitting datasets across variables for rechunking.
It also includes the new transforms SplitVariables() and ConsolidateVariables(), and exposes a number of internal helper functions as public APIs.