leap-stc / eNATL_feedstock

Apache License 2.0
0 stars 1 forks source link

Issue with chunks when combining #7

Open SammyAgrawal opened 1 week ago

SammyAgrawal commented 1 week ago

pangeo_forge_recipes/rechunking.py

raise ValueError("Cannot combine fragments because they do not form a regular hypercube.")

ValueError: Cannot combine fragments because they do not form a regular hypercube. [while running 'Create|OpenURLWithFSSpec|OpenWithXarray|Preprocess|StoreToZarr/StoreToZarr/Rechunk/MapTuple(combine_fragments)']

SammyAgrawal commented 1 week ago

Source Code raising the error in pangeo forge:

    # now we need to unstack the 1D concat dims into an ND nested data structure
    # first step is figuring out the shape
    dims_starts_sizes = [(
            dim.name,
            [index[dim].value for index in all_indexes],
            [ds.sizes[dim.name] for ds in all_dsets],
        )
        for dim in concat_dims]
[...]

starts_cube = [np.array(item[1]).reshape(shape) for item in dims_starts_sizes]
    sizes_cube = [np.array(item[2]).reshape(shape) for item in dims_starts_sizes]
    try:
        # reversing order is necessary here because _sort_by_speed_of_varying puts the
        # arrays into the opposite order as wanted by np.meshgrid
        starts = _invert_meshgrid(*starts_cube[::-1])[::-1]
        sizes = _invert_meshgrid(*sizes_cube[::-1])[::-1]
    except AssertionError:
        raise ValueError("Cannot combine fragments because they do not form a regular hypercube.")