NodeFileFilter fails with "assertion error" if a variable has an empty string for attribute

[zarzycki]

Netcdf 4.8.0 on MacOS.

Some WRF variables have attributes such as "stagger" -- for some runs (HF TGW, for example) this variable is just an empty string.

    float SLP(time, south_north, west_east) ;
        SLP:description = "Sea Level Pressure" ;
        SLP:units = "hPa" ;
        SLP:coordinates = "XLONG XLAT XTIME" ;
        SLP:stagger = "" ;
        SLP:MemoryOrder = "XYZ" ;
        SLP:FieldType = 104 ;

When running NFF with multiple variables passed in...

Ex:

${TEMPESTEXTREMESDIR}/bin/NodeFileFilter --in_nodefile 2005_test.txt --in_fmt "lon,lat,pres,wind,phis" --in_data_list "precip_file_list.txt" --out_data_list "precip_output_files.txt" --bydist 5.0 --var "SLP,U10,V10,OLR" --preserve "XLAT,XLONG" --regional --fillvalue "nan" --maskvar "mask" --latname "XLAT" --lonname "XLONG"

TE will fail with the following error:

Processing input (/Users/cmz5202/NetCDF/TGW/aux/wrfout_d01_2005-06-11_00:00:00_3hourly.aux.nc)
..EXCEPTION (/Users/cmz5202/Software/tempestextremes/src/base/NetCDFUtilities.cpp, Line 153) Assertion failure

If one deletes the empty attribute from all vars on the file, such as:

ncatted -O -a stagger,,d,, aux/wrfout_d01_2005-06-11_00\:00\:00_3hourly.aux.nc aux/wrfout_d01_2005-06-11_00\:00\:00_3hourly.aux.nc 

TE is successful.

[paullric]

So stagger seems to be a NetCDF “char” attribute with one value for each character of that string (this is different from the NetCDF string type which was introduced in NetCDF4). Apparently if that string is blank then the number of values is equal to zero, which I’d assumed was impossible.

I don’t immediately see any issue with removing that _ASSERT. Perhaps try that and see if it works?

~ Paul

On Feb 18, 2022, at 4:43 PM, zarzycki @.***> wrote:

Netcdf 4.8.0 on MacOS.

Some WRF variables have attributes such as "stagger" -- for some runs (HF TGW, for example) this variable is just an empty string.

float SLP(time, south_north, west_east) ; SLP:description = "Sea Level Pressure" ; SLP:units = "hPa" ; SLP:coordinates = "XLONG XLAT XTIME" ; SLP:stagger = "" ; SLP:MemoryOrder = "XYZ" ; SLP:FieldType = 104 ; When running NFF with multiple variables passed in...

Ex:

${TEMPESTEXTREMESDIR}/bin/NodeFileFilter --in_nodefile 2005_test.txt --in_fmt "lon,lat,pres,wind,phis" --in_data_list "precip_file_list.txt" --out_data_list "precip_output_files.txt" --bydist 5.0 --var "SLP,U10,V10,OLR" --preserve "XLAT,XLONG" --regional --fillvalue "nan" --maskvar "mask" --latname "XLAT" --lonname "XLONG" TE will fail with the following error:

Processing input (/Users/cmz5202/NetCDF/TGW/aux/wrfout_d01_2005-06-11_00:00:00_3hourly.aux.nc) ..EXCEPTION (/Users/cmz5202/Software/tempestextremes/src/base/NetCDFUtilities.cpp, Line 153) Assertion failure If one deletes the empty attribute from all vars on the file, such as:

ncatted -O -a stagger,,d,, aux/wrfout_d01_2005-06-11_00\:00\:00_3hourly.aux.nc aux/wrfout_d01_2005-06-11_00\:00\:00_3hourly.aux.nc TE is successful.

— Reply to this email directly, view it on GitHub https://github.com/ClimateGlobalChange/tempestextremes/issues/33, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABIVPZDU52AVT5FDGBTUAX3U33RSHANCNFSM5OZP2XRQ. Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub. You are receiving this because you are subscribed to this thread.

[zarzycki]

Yes, commenting out that line in NetCDFUtilities does avoid the issue.

I'm trying to think of edge cases where having that _ASSERT is helpful -- I suppose you could only call the _ASSERT on non-char attributes to handle this particular problem?

But for now, removing that is an alternative to using NCO to modify the files (have TE handle that would probably be preferable in the long run anyways...)

[paullric]

Removed _ASSERT in commit a361e4fb5d8606517d66233eba15132fefdfbed2