Conflicting dimension in newer xarray version

[CapeHobbit]

Not really an issue, just an FYI!

First; thanks for putting together this excellent toolkit! I have been using it for a couple of case studies for EUMETSAT (who I contract for; e.g. https://www.eumetsat.int/marine-heatwave-intensification-threatens-coral-reef-health), with associated Jupyter Notebooks.

However, I think some recent updates to the concat method in xarray break some of your methods. See here for an example. If I roll xarray back to 2022.3.0 everything is fine, but until they roll out a fix 2022.6.0 won't work and will give errors related to duplication of the ts.cell indices. You may want to update your build instructions in the interim. Hope this is useful.

Thanks again!

[florianboergel]

Same for me. I have not been able to solve the problem, however, downgrading to version 2022.3.0 helped.

[paolap]

Thanks for letting me know, I don't use it myself, and I'm drowning in work so didn't have time to check it. I will run some tests and see if I can fix it :-)

[paolap]

Thanks again, looks like the xarray bug has already been resolved, so I've temporarily updated the requirements files, hopefully they'll have a new release soon and then I'll check if this is still happening

[bnjmnr]

I stumbled into the same issue and after some digging, it seems that it comes from the xarray side and particularly in a difference of treatment between the methods mean and quantile. While the first one keeps all the coordinates, including the ones having only one element as is the case for the cell coordinate at that stage of the code, it is not the case for quantile which drops those coordinates for some unknown reason. This is the cause of an error when calling xr.concat at line 173 of the xmhw.py file, because there is no cell coordinate to concatenate anymore. So far, my workaround is to add a little method in the comprehensive list command of line 172 (thresh_results = [r[0] for r in results[0]]) to reassign the command: thresh_results = [r[0].assign_coords(r[1]['cell'].coords) for r in results[0]] It is ugly, but it works. Hopefully something better can be achieved, or the xarray methods will be homogenized soon. In any case, thanks a lot for doing this package, very complementary to the Oliver one.

[paolap]

thanks everyone, looks like I finally have a bit of time to look into this

[paolap]

After having a look, just re-assigning the coordinates doesn't work for me, as they aren't anymore a multi-index but. anomy object and I'm getting an error later when concatenating. After a bit of trial and error I managed to make it work by:

• not creating a multi-index when stacking the variable in land_check()
• re-assigning coordinates (but lat and lon rather than cell) as shown by @bnjmnr
• concatenating on "cell", resetting the index, finally unstacking

  thresh_results = [r[0].assign_coords(r[1].coords) for r in results[0]]
  ds["thresh"] = xr.concat(thresh_results, dim='cell')
  ds.thresh.name = "threshold"
  seas_results = [r[1] for r in results[0]]
  ds["seas"] = xr.concat(seas_results, dim='cell')
  ds.seas.name = "seasonal"
  dims = [k for k in ts.cell.coords.keys()]
  ds = ds.set_xindex(dims)
  ds = ds.unstack(dim='cell')

  This is working but I need to adapt a test before doing a new release. Hopefully getting rid of the multi-index might have the side effect of speeding up calculation too

[bnjmnr]

Great, thanks @paolap!

[florianboergel]

Thanks for the effort, @bnjmnr and @paolap.

[paolap]

I had to do a bit of wrangling to make sure the coordinates were treated correctly everywhere. Now all tests are passing, I also used to chance to remove some annoying pandas warnings. Finger crossed this is it! :-) There's a new release 0.9.0