Permute dimension when using YAXarray from Dataset and two input cubes

[dpabon]

I'm trying to use one of the variables on my cube to mask the other variables on the same cube. I included the dataset in the following link: test_cube.zarr.zip

test_cube = open_dataset("test_cube.zarr")

test_cube = test_cube.metrics_for_transitions

# creating a mask dataset per class transition.

function masking_fuction_continous(out, cube_cont_var, cube_to_be_masked; cont_threshold::Float32)
    #println(size(cube_cont_var))

    println("mask size:", size(cube_cont_var))
    println("Cube to be masked size:", size(cube_to_be_masked))
    masked_pixels = findall(<(cont_threshold), cube_cont_var)

    for i in masked_pixels

        cube_to_be_masked[i] = NaN

    end

    out = cube_to_be_masked

end

indims_1 = InDims("lat", "lon")
indims_2 = InDims("lat", "lon")

indims_total = (indims_1, indims_2)

outdims = OutDims("lat", "lon")

test_masked = mapCube(masking_fuction_continous, (test_cube[Differences = "coocurence"], test_cube), indims = (indims_total), outdims = outdims; cont_threshold = co_occurrence_thresholds)

The problem is that when I print the size of cube_cont_var and cube_to_be_masked I got

mask size:(181, 360)
Cube to be masked size:(360, 181)

I'm not certain why the dimensions' matrix is permuted just for one matrix. Am I missing something?

Thanks in advance for your help. Daniel

[meggart]

I think there are at least 2 things going on here. 1) it is impossible to modify input arrays. Although it looks like you can write into cube_to_be_masked it will never be modified. You would have to write into out and use this as a new data cube. 2) The permuted dimensions are really strange and I am still looking into this.

[meggart]

To be precise, to solve 1) it should be out .= cube_to_be_masked

[dpabon]

Hi Fabian,

Thanks for your help! yes, you're right. Please find a working version of the example above here:

function masking_fuction_continous(out, cube_cont_var, cube_to_be_masked; cont_threshold::Float32)

    #println("mask size:", size(cube_cont_var))
    #println("Cube to be masked size:", size(cube_to_be_masked))
    #println("out size: ", size(out))
    out .= NaN32
    #println(cube_to_be_masked)
    out .= transpose(cube_to_be_masked)
    if (!all(isnan, cube_cont_var)) | (!all(ismissing, cube_cont_var))

        masked_pixels = findall(x->x<cont_threshold, cube_cont_var)
        for i in masked_pixels
            #println(temporal[i])
            out[i] = NaN32

        end
    end
    #println(out)

end

indims_1 = InDims("lon", "lat", filter=YAXArrays.DAT.NoFilter())
indims_2 = InDims("lon", "lat", filter=YAXArrays.DAT.NoFilter())

indims_total = (indims_1, indims_2)

outdims = OutDims("lon", "lat")

test_masked = mapCube(masking_fuction_continous, (test_cube[Differences = "coocurence"], test_cube), indims = (indims_total), outdims = outdims; cont_threshold = Float32(0.2))

This new version produces outputs for the first time steps, but the co-occurrence variable is somehow replaced by one of the other two variables.

[dpabon]

I tested the branch with the fix, and it solves the problem.

[lazarusA]

let's keep this one open, until https://github.com/JuliaDataCubes/YAXArrays.jl/pull/201 is merged. @meggart ?