Error about missing metrics in a derived variable

[tlsw231]

1. I have a dataset from an ocean model (ROMS).
2. I use resample to compute daily-averaged values of two variables, do some basic operations to remove the monthly climatology and compute the covariance of the anomalies.
3. When I use xroms.ddxi to get the x-derivative of the covariance, I get an error message complaining about the absence of metrics.

I have uploaed a copy of the notebook (in html format) which shows the error message here (have omitted some initial cells to create the list of filenames, etc.)

The attached notebook shows the grid attribute is present for the variable uT_atu so I am not sure why I am getting that error message.

[tlsw231]

I converted the array into a dataset before repeating step 3 (refer previous message). Still get the same error. [Edit: I had to manually add a set of variables (xroms_vars and zeta) into the dataset to ensure the xroms.roms_dataset step works.)

ds_derived = uT_atu.to_dataset(name='uT_atu')
for key in xroms_vars:
    ds_derived[key] = ds0[key]

ds_derived['zeta']=ds0['zeta']

ds_derived,grid_derived = xroms.roms_dataset(ds_derived)

There are also a couple of other things that are a bit puzzling to me. The xroms.ddxi command works (as it should) on a standard variable (u) from the dataset.

1. Unless I specify scoord='s_rho', the result of xroms.ddxi applied to u ends up with an s_w vertical coordinate. Not sure why.
2. The output from xroms.ddxi of u ends up doubling the size of the original array when I would have thought they should be nearly the same size.

[tlsw231]

It looks like xroms.ddz(uT_atu,grid_derived) works on the derived variable but both xroms.ddxi and xroms.ddeta fail with the error message described earlier.

[kthyng]

@tlsw231 Thanks for bringing this over here from Pangeo's discourse, in particular because it helped me realize that I didn't have my notifications set up to watch this repo!

I haven't had a chance to try to reproduce this error and dig into it — I'm currently on maternity leave with inconsistent time at my computer! I intend to do so when I can though.

[tlsw231]

I think I figured out what was the problem, took some wading through.

1. I was using resample (step 2 in my first post) which drops the time-varying coordinate z_rho_u but retains the time-invariant coordinate z_rho_u0. This was causing the following line in xroms/utilities.py to output z_rho_u0:

z_coord_name = coords[[coord[:2] == "z_" for coord in coords].index(True)],

which was then triggering errors in the code downstream.

2. The error vanishes and I get meaningful output if I use resample and reset_coords(drop=True) in the same step. The reset_coords ends up dropping both z_rho_u0 and z_rho_u.

So, it looks like if there are no coordinates beginning with z_ the xroms.ddxi step goes through. But if there is only one coordinate beginning with z_ and it happens to be z_rho_u0 then I face the issue outlined above. More generally, this problem might arise after any calculation that ends up dropping the time-varying z_ coordinates while retaining the time-invariant z_ coordinates.

While reset_coords appears to fix the issue, I have to wonder how accurate the xroms.ddxi calculation will be given it is taking place without any knowledge of the underlying vertical grid. Will it not be less accurate compared to the case where I apply it directly to a variable in the dataset, e.g., ds['u'], which has both z_rho_u and z_rho_u0?

[tlsw231]

Even when both z_rho_u0 and z_rho_u are present, this line will return the former if it appears earlier in the sequence of coordinates.

z_coord_name = coords[[coord[:2] == "z_" for coord in coords].index(True)]

Can this be changed to the following?

z_coord_name = coords[ ([coord[:2] == "z_" and "0" not in coord) for coord in coords].index(True)]

[kthyng]

Hi @tlsw231! I'm setting up to work on xroms again over time. I think I'll start with improving some of the basics like packaging before diving back into the details, but I intend to get back to you on this soon. Thanks for the issue report.

[kthyng]

Hi @tlsw231! I've recently put out a new version to PyPI (v0.2.4, still coming through on conda-forge). Please check to see if this issue is still present in the new version so that over time we can work to address these. Thanks!

[tlsw231]

Hi @kthyng, thanks for the update! I have moved on to a different line of work where I don't use much computational resources but hope somebody else can check if the above issue persists.

[kthyng]

@tlsw231 Fair enough, thank you for your response and good luck in your new line of work! I'll close this for now.