Error while running the model

[geethar123]

Hi Sir, I have used pyClim SDM for running Preprocess of the model. I am getting the following error:

grids.association() starts tasmax grids.association nearest 0.0 % tasmax grids.association nearest 54.6448087431694 % grids.association() starts tasmax grids.association bilinear 0.0 % TypeError: only size-1 arrays can be converted to Python scalars The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/mnt/d/pyClim-SDM-master/src/.tmp_main.py", line 13, in main() File "/mnt/d/pyClim-SDM-master/src/.tmp_main.py", line 9, in main preprocess.preprocess() File "/mnt/d/pyClim-SDM-master/src/../lib/preprocess.py", line 43, in preprocess common() File "/mnt/d/pyClim-SDM-master/src/../lib/preprocess.py", line 59, in common grids.association(interp_mode, targetVar) File "/mnt/d/pyClim-SDM-master/src/../lib/grids.py", line 169, in association df_association['i1'].iloc[ipoint] = ilatUp File "/mnt/d/anaconda/anaconda/envs/GEETHA/lib/python3.10/site-packages/pandas/core/indexing.py", line 716, in setitem iloc._setitem_with_indexer(indexer, value, self.name) File "/mnt/d/anaconda/anaconda/envs/GEETHA/lib/python3.10/site-packages/pandas/core/indexing.py", line 1690, in _setitem_with_indexer self._setitem_single_block(indexer, value, name) File "/mnt/d/anaconda/anaconda/envs/GEETHA/lib/python3.10/site-packages/pandas/core/indexing.py", line 1938, in _setitem_single_block self.obj._mgr = self.obj._mgr.setitem(indexer=indexer, value=value) File "/mnt/d/anaconda/anaconda/envs/GEETHA/lib/python3.10/site-packages/pandas/core/internals/managers.py", line 337, in setitem return self.apply("setitem", indexer=indexer, value=value) File "/mnt/d/anaconda/anaconda/envs/GEETHA/lib/python3.10/site-packages/pandas/core/internals/managers.py", line 304, in apply applied = getattr(b, f)(**kwargs) File "/mnt/d/anaconda/anaconda/envs/GEETHA/lib/python3.10/site-packages/pandas/core/internals/blocks.py", line 955, in setitem values[indexer] = value ValueError: setting an array element with a sequence.

Please help me to sort it out

[ahernanzl]

Hi, are you using the toy datasets included with the software or your own datasets?

[geethar123]

My own dataset sir.

[geethar123]

[ahernanzl]

Try the toy datasets. If the error takes place only with your own dataset, make sure it follows the format of the toy datasets.

[geethar123]

Thank you sir. The problem is in : df_association['i1'].iloc[ipoint] = ilatUp Will you explain the i1 and iloc[ipoint] in the code ((grids.py)- line 169.

[ahernanzl]

But have you tried the datasets included?

[geethar123]

Yes sir. Its working. I am unable to find what's the problem in my data. Thats y

[ahernanzl]

Can you print ilatUp and see what value it contains?

[geethar123]

The output of ilatUp is empty array sir ie [ ].

[ahernanzl]

Ok, I see. A single integer was expected for ilatUp. Print latDown, latUp, grid_res, pred_lats and hres_lat, please.

[geethar123]

I am attaching the screenshots for hres_lat and pred_lat. latUp =8.690529730584887 latDown = 8.440529730584887 grid_res =0.25

[ahernanzl]

If your grid_res is 0.25, that is what is expected in the reanalysis netCDFs (2D low resolution regular grid). And as for hres, a 1D list of points is expected. It seems to me that your data do not fotllow that format, right? If your hres is 2D you need to flatten it. Let me know if you get it

[geethar123]

Sir, I have seen your reply in issue #4 that reanalysis and model resolution should be same. So, I have modified the reanalysis data to models' resolution. Please clarify it sir. Also, I am following the same data format as hres sample data.

[ahernanzl]

Yes, reanalysis and models should share resolution. That is grid_res. And if you folow the format of the toy datasets included with the software for reanalysis, models and hres, everything should work fine. Hope it works for you

[geethar123]

Sir, my requirement is to downscale the GCM from 100 km horizontal resolution to 25 km using statistical downscaling. I have the observation data and reanalysis data at 25 km horizontal resolution and model data is in 100 km resolution. I am not clear that how can a grid_res, reanalysis and model data can be in same resolution. If model and grid_res is same, what is the benefit of downscaling. Please clarify it sir.

[ahernanzl]

Hi, you need three datasets: hres (high resolution observations, 1D), reanalysis (low resolution predictors for calibration, 2D) and models (low resolution predictors for downscaling, 2D).

[geethar123]

Sorry sir, I am not clear. I have hres data of high resolution(1D), reanalysis data of 25 km horizontal resolution and model data of 100 km resolution. I want the downscaled output to be of 0.25 grid resolution. Can I use pyClim - SDM for this. If yes, kindly suggest what can be the resolution of reanalysis and model data

[ahernanzl]

Hi, your downscaled output will correspond to your hres data (observations). And in order to train your methods you need predictors from a reanalysis at the same resolution of your models. So you need to regrid your reanalysis to 100 km.

[geethar123]

Hi Sir, Thanks a lot for your suggestions. I am able to run the model. The output netcdf file created while running projection has different structure. It does not have latitude, longitude and time as dimension. Kindly help to sort out this issue.

[ahernanzl]

Hello, I'm glad to hear you have been able to run the model. The lack of longitude dimension is an error that have been fixed, so one option is to update the software (and reprocess). Another option is to convert netCDFs to ASCII. Hope that helps!

[geethar123]

Hi Sir, Thank you for the immediate reply. I have downloaded the latest pyClim SDM software and executed the sample data. Still, i am getting the netcdf with same structure. Also, it will be difficult to process the large data if it is ASCII format. Kindly suggest the solution for this issue.

[ahernanzl]

Hello, With the latest version, netCDF outputs shoud contain lat,lon,time dimensions, but as an unstructured list of points. netCDFs are CF compliant, and they should be easily readable with python or R for example. On the other hand, many netCDF viewers do not support this type of files, but only 2D regular grids.