climpact and climdex.pcic.ncdf do not compute SPI and SPEI

[user241276]

hello @heroldn @jmccomb @lisavalexander,

I am using both climpact and climdex.pcic.ncdf for computing indices for CMIP6 data. My input files are netcdfs with tasmax, tasmin and pr variables from 1950 to 2100.

The computation of the indices works fine but not for the SPI and SPEI indices.

Please see below an example if I do cdo info on one SPEI output (same applies for SPI):

[XXPP@log test]$ cdo info spei_MON_cmip6_ssp126_ACCESS-CM2_1950-2100.nc
Warning (cdfScanVarAttr): Time must be the first dimension! Unsupported array structure, skipped variable spei!
Warning (cdf_set_var): Inconsistent variable definition for spei!
Warning (cdfInqContents): No data arrays found!
cdo    info: Open failed on >spei_MON_cmip6_ssp126_ACCESS-CM2_1950-2100.nc<
             Unsupported file structure

Looks like the aren't any data in the netcdf file.

What am I missing?

Thanks

[heroldn]

You'll probably find there is data in those files, unfortunately they are not CF compliant so CDO might report an error.

Do an ncdump -h on them to confirm. This happens because the 3 SPI time scales are stored along an additional dimension, so the data is stored something like scale x time x lat x lon.

You can use NCO to extract the scale to its own file, from there it should be CDO friendly.

E.g. ncks -d scale,0 -v spi climpact_spi_file.nc out.nc

The above line will extract SPI3, using "scale,1" should extract SPI6, and "scale,2" should extract SPI12. Haven't tested any of this and YMMV. Let me know if that doesn't work though.

On 17/11/2021 20:35 user241276 @.***> wrote:

hello @heroldn https://github.com/heroldn @jmccomb https://github.com/jmccomb @lisavalexander https://github.com/lisavalexander ,

I am using both climpact and climdex.pcic.ncdf for computing indices for CMIP6 data. My input files are netcdfs with tasmax, tasmin and pr variables from 1950 to 2100.

The computation of the indices works fine but not for the SPI and SPEI indices.

Please see below an example if I do cdo info on one SPEI output (same applies for SPI):

@.*** test]$ cdo info spei_MON_cmip6_ssp126_ACCESS-CM2_1950-2100.nc Warning (cdfScanVarAttr): Time must be the first dimension! Unsupported array structure, skipped variable spei! Warning (cdf_set_var): Inconsistent variable definition for spei! Warning (cdfInqContents): No data arrays found! cdo info: Open failed on >spei_MON_cmip6_ssp126_ACCESS-CM2_1950-2100.nc< Unsupported file structure

Looks like the aren't any data in the netcdf file.

What am I missing?

Thanks

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ARCCSS-extremes/climpact/issues/91 , or unsubscribe https://github.com/notifications/unsubscribe-auth/ACW3FAMZQCV64ZMUEABJ5STUMN45BANCNFSM5IGQJK3A . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub .

[user241276]

hello, thanks a lot for the support.

I tried with ncks -d scale,0 -v spi climpact_spi_file.nc test.nc but I still get an empty output.

see attached a ncview screenshot of test.nc

a cdo info on the same file:

[XXxDD@log dir]$ cdo info test.nc
Warning (cdfScanVarAttr): Time must be the first dimension! Unsupported array structure, skipped variable spi!
Warning (cdf_set_var): Inconsistent variable definition for spi!
Warning (cdfInqContents): No data arrays found!
cdo    info: Open failed on >test.nc<
             Unsupported file structure

I also tested it with a longer bas-range of 30 years but results are the same (now I am using 20y base-range).

[heroldn]

Go forward in time past the first 2 time steps for SPI3. Past the first 5 steps for SPI6, and 11 time steps for SPI12. These first time steps don't have any data since the SPI represents an accumulation of precipitation over the past 3/6/12 months.

On 19/11/2021 20:24 user241276 @.***> wrote:

hello, thanks a lot for the support.

I tried with NCO but I still get an empty output.

see attached a ncview screenshot of SPI3 [Screenshot 2021-11-19 at 10 50 54] https://user-images.githubusercontent.com/31700697/142602536-e09abf62-52e0-4f61-b351-98b85d3a78b8.png

a cdo info on the same file:

@.*** dir]$ cdo info test.nc Warning (cdfScanVarAttr): Time must be the first dimension! Unsupported array structure, skipped variable spi! Warning (cdf_set_var): Inconsistent variable definition for spi! Warning (cdfInqContents): No data arrays found! cdo info: Open failed on >test.nc< Unsupported file structure

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ARCCSS-extremes/climpact/issues/91#issuecomment-973920501 , or unsubscribe https://github.com/notifications/unsubscribe-auth/ACW3FAPIQIED2RPOK44W743UMYNG7ANCNFSM5IGQJK3A . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub .

[user241276]

thanks. It makes sense :)

I tried an example with SPI3 and cdo and it does not work:

[aaaa@log dir]$ cdo seltimestep,3/30 test.nc out.nc
Warning (cdfScanVarAttr): Time must be the first dimension! Unsupported array structure, skipped variable spi!
Warning (cdf_set_var): Inconsistent variable definition for spi!
Warning (cdfInqContents): No data arrays found!
cdo    seltimestep: Open failed on >test.nc<
                    Unsupported file structure

Then tried the same with NCO but when plotting with ncview I only have one SPI3 value:

[aaaa@log dir]$ ncks -d time,3,30 test.nc out.nc
[aaaa@log dir]$ ncview out.nc

Is it 'normal' that ncview cannot plot the SPI? How can I access the data inside test.nc?

Thanks

[heroldn]

It looks like you have more than one SPI3 value. Are you sure you're looking at the time dimension? It goes from 36104.5 to 36914.5. You don't need to extract those timesteps with CDO/NCO either, just use ncview and move forward in the time dimension by clicking the box I've circled in red.

[user241276]

great. it works :) thank you

[user241276]

p.s. to make the file "test.nc" work with CDO, the dimension "scale" needs to be removed from SPI3, SPI6 or/and SPI12. This can be done with NCO:

ncwa -a scale test.nc out.nc

best

[heroldn]

Yes true. Good to hear you can access the data.