search

monocongo / climate_indices

Climate indices for drought monitoring
https://monocongo.github.io/climate_indices/
Other
343 stars 168 forks source link

Invalide periodicity argument #371

Closed Romaysa closed 4 years ago

Romaysa commented 4 years ago

Sir. I run the bellow python code to get a 3 month scales SPI for my study aera.

python process_grid.py --index spi --periodicity monthly --netcdf_precip ../example_inputs/IMERG_concat_ncpdq.nc4 -- var_name_precip precipitation --output_file_base ../results/IMERG --scales 3 - -calibration_start_year 2000 --calibration_end_year 2019

the Periodicity argument is giving an error about monthly data. could you please check the proccess_grid.py

ValueError: Invalid periodicity argument: monthly

"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "process_grid.py", line 1137, in args.scales) File "process_grid.py", line 1062, in process_grid grid_processor.run() File "process_grid.py", line 332, in run result.get() File "C:\Users\ibrah\Anaconda3\lib\multiprocessing\pool.py", line 657, in get raise self._value ValueError: Invalid periodicity argument: monthly##

Regards

monocongo commented 4 years ago

Can you please post/link your dataset so I can attempt to reproduce the error?

You may be using a previous version of the package, as process_grid.py no longer exists.

Which version are you using? How did you install the package?

Romaysa commented 4 years ago

Respected Sir, I downloaded the process_grid.py from this link. https://www.drought.gov/drought/climate-and-drought-indices-python

I used GIT BASH and configured Anaconda3 and run the program.

The version of climate indices which i downloaded from above link was giving error for this test python setup.py test. So i download it from your Github and run the test and it was all okay and run without any test error.

The difference between the Github version and the downloaded one from the link above explaind in the attached pictures, some of the folders are not in github climate indices version which is avilable in the other one.

kindly find the attached dataset and the traning manual i used to run the climate indices.

github_version indices_python IMERG.zip

Regards

Romaysa commented 4 years ago

Sir, I forgot to attached the traning manule i used for this analyses, could you please have a look.

IMERG_Week2_FINAL.pdf

samburu commented 4 years ago

Hey @Romaysa, If you don't mind using geotiff, I'm more than happy to help with the code for converting netcdf to geotiff and calculating the 3 month SPI. The steps are:

  1. Convert netcdf to geotiff
  2. Read geotiff as numpy array
  3. Calculate SPI
  4. Save SPI array as geotiff
Romaysa commented 4 years ago

Can you please post/link your dataset so I can attempt to reproduce the error? Sir, the dataset which is used is attached here as IMERG zip folder. IMERG (1).zip

You may be using a previous version of the package, as process_grid.py no longer exists. the package was download using the link below https://www.drought.gov/drought/climate-and-drought-indices-python

Which version are you using? How did you install the package? i followed the traning manual of AREST, which i attached here. IMERG_Week2_FINAL.pdf

Regards

Romaysa commented 4 years ago

Hey @Romaysa, If you don't mind using geotiff, I'm more than happy to help with the code for converting netcdf to geotiff and calculating the 3 month SPI. The steps are:

  1. Convert netcdf to geotiff
  2. Read geotiff as numpy array
  3. Calculate SPI
  4. Save SPI array as geotiff

Respected, if i coulden't solve it diffinetly i will ask you about the code please. Regards

Romaysa commented 4 years ago

Can you please post/link your dataset so I can attempt to reproduce the error?

You may be using a previous version of the package, as process_grid.py no longer exists.

Which version are you using? How did you install the package?

https://github.com/monocongo/climate_indices/issues/371#issuecomment-588251581 regards

Romaysa commented 4 years ago

Can you please post/link your dataset so I can attempt to reproduce the error?

You may be using a previous version of the package, as process_grid.py no longer exists.

Which version are you using? How did you install the package?

Sır. will i expect any answer about the problem i sent?

Regards

monocongo commented 4 years ago

Hi @Romaysa, sorry for the delayed response.

I have alerted the NIDIS group as to the problems you have experienced as a result of the out-of-date version of the code which they are providing on their site.

It appears that you have overcome your issue by downloading and using the latest version from the master branch of this repository. Please advise if you are still having issues.

My advice is to not use the code provided by NIDIS but instead use the code provided here and to follow the directions provided in the documentation.

Also, I recommend contacting the authors of the IMERG tutorial with your result and recommendation that they update their document to reference this repository for the software since the version provided by NIDIS is out-of-date and untested/unreliable.

Romaysa commented 4 years ago

Hi @Romaysa, sorry for the delayed response.

I have alerted the NIDIS group as to the problems you have experienced as a result of the out-of-date version of the code which they are providing on their site.

It appears that you have overcome your issue by downloading and using the latest version from the master branch of this repository. Please advise if you are still having issues.

My advice is to not use the code provided by NIDIS but instead use the code provided here and to follow the directions provided in the documentation.

Also, I recommend contacting the authors of the IMERG tutorial with your result and recommendation that they update their document to reference this repository for the software since the version provided by NIDIS is out-of-date and untested/unreliable.

Respected Sir, I followed the updated instruction i copied my nc data in the folder called data but i got these errors and warnings.

C:\Users\ibrah\Anaconda3\lib\site-packages\climate_indices-1.0.9-py3.7.egg\climate_indices\compute.py:403: NumbaWarning: Compilation is falling back to object mode WITHOUT looplifting enabled because Function "_pearson_fit_ufunc" failed type inference due to: Invalid use of type(CPUDispatcher(<function _pearson3cdf at 0x000002CE3836F438>)) with parameters (float32, list(float32))

File "....\Anaconda3\lib\site-packages\climate_indices-1.0.9-py3.7.egg\climate_indices\compute.py", line 446: <source missing, REPL/exec in use?>

C:\Users\ibrah\Anaconda3\lib\site-packages\numba\compiler.py:742: NumbaWarning: Function "_pearson_fit_ufunc" was compiled in object mode without forceobj=True.

File "....\Anaconda3\lib\site-packages\climate_indices-1.0.9-py3.7.egg\climate_indices\compute.py", line 403: <source missing, REPL/exec in use?>

self.func_ir.loc)) C:\Users\ibrah\Anaconda3\lib\site-packages\numba\compiler.py:751: NumbaDeprecationWarning: Fall-back from the nopython compilation path to the object mode compilation path has been detected, this is deprecated behaviour.

For more information visit http://numba.pydata.org/numba-doc/latest/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit

File "....\Anaconda3\lib\site-packages\climate_indices-1.0.9-py3.7.egg\climate_indices\compute.py", line 403: <source missing, REPL/exec in use?>

warnings.warn(errors.NumbaDeprecationWarning(msg, self.func_ir.loc)) C:\Users\ibrah\Anaconda3\lib\site-packages\climate_indices-1.0.9-py3.7.egg\climate_indices\compute.py:403: NumbaWarning: Compilation is falling back to object mode WITHOUT looplifting enabled because Function "_pearson_fit_ufunc" failed type inference due to: Invalid use of type(CPUDispatcher(<function _pearson3cdf at 0x000002CE3836F438>)) with parameters (float64, list(float64))

File "....\Anaconda3\lib\site-packages\climate_indices-1.0.9-py3.7.egg\climate_indices\compute.py", line 446: <source missing, REPL/exec in use?>

C:\Users\ibrah\Anaconda3\lib\site-packages\numba\compiler.py:742: NumbaWarning: Function "_pearson_fit_ufunc" was compiled in object mode without forceobj=True.

File "....\Anaconda3\lib\site-packages\climate_indices-1.0.9-py3.7.egg\climate_indices\compute.py", line 403: <source missing, REPL/exec in use?>

self.func_ir.loc)) C:\Users\ibrah\Anaconda3\lib\site-packages\numba\compiler.py:751: NumbaDeprecationWarning: Fall-back from the nopython compilation path to the object mode compilation path has been detected, this is deprecated behaviour.

For more information visit http://numba.pydata.org/numba-doc/latest/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit

File "....\Anaconda3\lib\site-packages\climate_indices-1.0.9-py3.7.egg\climate_indices\compute.py", line 403: <source missing, REPL/exec in use?>

warnings.warn(errors.NumbaDeprecationWarning(msg, self.func_ir.loc)) 2020-02-28 13:48:18 INFO Start time: 2020-02-28 13:48:18.107610 2020-02-28 13:48:18 ERROR Failed to complete Traceback (most recent call last): File "C:\Users\ibrah\Anaconda3\lib\site-packages\climate_indices-1.0.9-py3.7.egg\climate_indices__main__.py", line 1588, in main type=compute.Periodicity.from_string, File "C:\Users\ibrah\Anaconda3\lib\enum.py", line 349, in getattr raise AttributeError(name) from None AttributeError: from_string Traceback (most recent call last): File "C:\Users\ibrah\Anaconda3\Scripts\process_climate_indices-script.py", line 11, in load_entry_point('climate-indices==1.0.9', 'console_scripts', 'process_climate_indices')() File "C:\Users\ibrah\Anaconda3\lib\site-packages\climate_indices-1.0.9-py3.7.egg\climate_indices__main.py", line 1588, in main File "C:\Users\ibrah\Anaconda3\lib\enum.py", line 349, in getattr__ raise AttributeError(name) from None AttributeError: from_string (indices_env)

Regards

monocongo commented 4 years ago

Please advise with the command you are using and a link to the data you are using as input. Also please advise as to which version of numba you have installed. For example please provide the result of this command:

$ conda list numba

In the interim, you may try setting this environment variable to disable numba's just-in-time compilation, which I assume will get you past this issue which appears to be numba related:

$ export NUMBA_DISABLE_JIT=1
Romaysa commented 4 years ago

conda list numba Sir, this is the NUMA version packages in environment at C:\Users\ibrah\Anaconda3: #

Name Version Build Channel

numba 0.45.1 py37hf9181ef_0

I paste this command as will in time of compilation but still got these errors

C:\Users\ibrah\Anaconda3\lib\site-packages\climate_indices-1.0.9-py3.7.egg\climate_indices\compute.py:403: NumbaWarning: Compilation is falling back to object mode WITHOUT looplifting enabled because Function "_pearson_fit_ufunc" failed type inference due to: Untyped global name '_pearson3cdf': cannot determine Numba type of <class 'function'>

File "....\Anaconda3\lib\site-packages\climate_indices-1.0.9-py3.7.egg\climate_indices\compute.py", line 446: <source missing, REPL/exec in use?>

C:\Users\ibrah\Anaconda3\lib\site-packages\numba\compiler.py:742: NumbaWarning: Function "_pearson_fit_ufunc" was compiled in object mode without forceobj=True.

File "....\Anaconda3\lib\site-packages\climate_indices-1.0.9-py3.7.egg\climate_indices\compute.py", line 403: <source missing, REPL/exec in use?>

self.func_ir.loc)) C:\Users\ibrah\Anaconda3\lib\site-packages\numba\compiler.py:751: NumbaDeprecationWarning: Fall-back from the nopython compilation path to the object mode compilation path has been detected, this is deprecated behaviour.

For more information visit http://numba.pydata.org/numba-doc/latest/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit

File "....\Anaconda3\lib\site-packages\climate_indices-1.0.9-py3.7.egg\climate_indices\compute.py", line 403: <source missing, REPL/exec in use?>

warnings.warn(errors.NumbaDeprecationWarning(msg, self.func_ir.loc)) C:\Users\ibrah\Anaconda3\lib\site-packages\climate_indices-1.0.9-py3.7.egg\climate_indices\compute.py:403: NumbaWarning: Compilation is falling back to object mode WITHOUT looplifting enabled because Function "_pearson_fit_ufunc" failed type inference due to: Untyped global name '_pearson3cdf': cannot determine Numba type of <class 'function'>

File "....\Anaconda3\lib\site-packages\climate_indices-1.0.9-py3.7.egg\climate_indices\compute.py", line 446: <source missing, REPL/exec in use?>

2020-03-02 11:52:11 INFO Start time: 2020-03-02 11:52:11.804402 2020-03-02 11:52:11 ERROR Failed to complete Traceback (most recent call last): File "C:\Users\ibrah\Anaconda3\lib\site-packages\climate_indices-1.0.9-py3.7.egg\climate_indices__main__.py", line 1588, in main type=compute.Periodicity.from_string, File "C:\Users\ibrah\Anaconda3\lib\enum.py", line 349, in getattr raise AttributeError(name) from None AttributeError: from_string Traceback (most recent call last): File "C:\Users\ibrah\Anaconda3\Scripts\process_climate_indices-script.py", line 11, in load_entry_point('climate-indices==1.0.9', 'console_scripts', 'process_climate_indices')() File "C:\Users\ibrah\Anaconda3\lib\site-packages\climate_indices-1.0.9-py3.7.egg\climate_indices__main.py", line 1588, in main File "C:\Users\ibrah\Anaconda3\lib\enum.py", line 349, in getattr__ raise AttributeError(name) from None AttributeError: from_string (indices_env)

monocongo commented 4 years ago

The above shows that you have numba version 0.45.1 installed.

Please also list the command you are using to run the indices calculation, as well as a link to the input data so I can attempt to duplicate the issue.

What was the result of disabling the numba JIT? Did that fix anything? I assume that this is a reasonable short-term solution as this issue appears to be numba related.

Romaysa commented 4 years ago

The above shows that you have numba version 0.45.1 installed.

Please also list the command you are using to run the indices calculation, as well as a link to the input data so I can attempt to duplicate the issue.

What was the result of disabling the numba JIT? Did that fix anything? I assume that this is a reasonable short-term solution as this issue appears to be numba related.

Sir, here is the link for the data.

https://romaysa.github.io/climateindices-romasa/

and i run this code

process_climate_indices --index spi --periodicity monthly --netcdf_precip /data/IMERG_concat_ncpdq.nc4 --var_name_precip prcp --output_file_base results/nclimgrid_lowres --scales 6 12 --calibration_start_year 2000 --calibration_end_year 2019 --multiprocessing all also i attached the data here in this email if the link dosen't work. data.zip

all the years are cancanicated to IMERG_concat_ncpdq.nc4.

Regards

Romaysa commented 4 years ago

Sir, i am still waiting for an answer hope you are doing well.

Regards

On Mon, Mar 2, 2020 at 7:07 PM James Adams notifications@github.com wrote:

The above shows that you have numba version 0.45.1 installed.

Please also list the command you are using to run the indices calculation, as well as a link to the input data so I can attempt to duplicate the issue.

What was the result of disabling the numba JIT? Did that fix anything? I assume that this is a reasonable short-term solution as this issue appears to be numba related.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/monocongo/climate_indices/issues/371?email_source=notifications&email_token=AGLXOFJLRS7BRW4ZTAOD4PLRFPKUXA5CNFSM4KXVD2XKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOENP4KKQ#issuecomment-593478954, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGLXOFJ4HESBWMMTNUNAWF3RFPKUXANCNFSM4KXVD2XA .

--

Ahmad Shah "Frahmand"

Ph.D Candidate

Geomatics Engineering Department, Yildiz Technical University, Istanbul, Turkey

ahmad.frahmand@gmail.com ahmad.frahmand@gmail.com

University official email: f5013021@std.yildiz.edu.tr

Skype: ah_frahmand_india

Mobile phone: 00905469743785

monocongo commented 4 years ago

I have attempted to reproduce the error using the dataset and command you provided above.

The first issue is that in the example command you are using the name "prcp" as the precipitation variable name, whereas the actual precipitation variable name in the file is "precipitation". This may be the cause of the attribute error you were seeing, I'm not sure.

Once I modified the command to use the correct variable name I then came across the issue of an unsupported unit being used -- the units for the data is "mm/hr" which indicates hourly data, and as such is unsupported. This is described in more detail in issue #363

I updated the precipitation dataset to use the correct units for the data, like so:

$ ncatted -a units,precipitation,modify,c,'mm' IMERG_concat_ncpdq.nc4 IMERGE_monthly_precip.nc

This fixed dataset is now valid input for the processing script and it completed without issue:

$ process_climate_indices --index spi --periodicity monthly --netcdf_precip IMERGE_monthly_precip.nc --var_name_precip precipitation --output_file_base IMERG --scales 6 12 --calibration_start_year 2000 --calibration_end_year 2019 --multiprocessing all
2020-03-09  10:58:37 INFO Start time:    2020-03-09 10:58:37.120279
2020-03-09  10:58:37 INFO Computing 6-month SPI/Pearson
2020-03-09  10:58:50 INFO Computing 6-month SPI/Gamma
2020-03-09  10:58:59 INFO Computing 12-month SPI/Pearson
2020-03-09  10:59:13 INFO Computing 12-month SPI/Gamma
2020-03-09  10:59:22 INFO End time:      2020-03-09 10:59:22.717935
2020-03-09  10:59:22 INFO Elapsed time:  0:00:45.597656

In summary the casues of error in this instance were 1) using an incorrect variable name at the command line, and 2) having an unsupported/invalid units for the precipitation variable.

Please let me know if this doesn't solve your issue. Good luck!

Romaysa commented 4 years ago

I have attempted to reproduce the error using the dataset and command you provided above.

The first issue is that in the example command you are using the name "prcp" as the precipitation variable name, whereas the actual precipitation variable name in the file is "precipitation". This may be the cause of the attribute error you were seeing, I'm not sure.

Once I modified the command to use the correct variable name I then came across the issue of an unsupported unit being used -- the units for the data is "mm/hr" which indicates hourly data, and as such is unsupported. This is described in more detail in issue #363

I updated the precipitation dataset to use the correct units for the data, like so:

$ ncatted -a units,precipitation,modify,c,'mm' IMERG_concat_ncpdq.nc4 IMERGE_monthly_precip.nc

This fixed dataset is now valid input for the processing script and it completed without issue:

$ process_climate_indices --index spi --periodicity monthly --netcdf_precip IMERGE_monthly_precip.nc --var_name_precip precipitation --output_file_base IMERG --scales 6 12 --calibration_start_year 2000 --calibration_end_year 2019 --multiprocessing all
2020-03-09  10:58:37 INFO Start time:    2020-03-09 10:58:37.120279
2020-03-09  10:58:37 INFO Computing 6-month SPI/Pearson
2020-03-09  10:58:50 INFO Computing 6-month SPI/Gamma
2020-03-09  10:58:59 INFO Computing 12-month SPI/Pearson
2020-03-09  10:59:13 INFO Computing 12-month SPI/Gamma
2020-03-09  10:59:22 INFO End time:      2020-03-09 10:59:22.717935
2020-03-09  10:59:22 INFO Elapsed time:  0:00:45.597656

In summary the casues of error in this instance were 1) using an incorrect variable name at the command line, and 2) having an unsupported/invalid units for the precipitation variable.

Please let me know if this doesn't solve your issue. Good luck!

Respected Sir,

first of thanks alot for your time. but still i am getting the error bellow. i even download the git version of climate indices newly and followed again the instruction from the documentation. i changed the unit from mm per hour to monthly based on the code you sent. but still i am getting this error sir. kindly have a look.

process_climate_indices --index spi --periodicity monthly --netcdf_precip IMERGE_monthly_precip.nc --var_name_precip precipitation --output_file_base IMERG --scales 6 12 --calibration_start_year 2000 --calibration_end_year 2019 --multiprocessing all C:\Users\ibrah\Anaconda3\lib\site-packages\climate_indices-1.0.9-py3.7.egg\climate_indices\compute.py:403: NumbaWarning: Compilation is falling back to object mode WITHOUT looplifting enabled because Function "_pearson_fit_ufunc" failed type inference due to: Untyped global name '_pearson3cdf': cannot determine Numba type of <class 'function'>

File "C:\Users\ibrah\Anaconda3\lib\site-packages\climate_indices-1.0.9-py3.7.egg\climate_indices\compute.py", line 446: <source missing, REPL/exec in use?>

C:\Users\ibrah\Anaconda3\lib\site-packages\numba\compiler.py:742: NumbaWarning: Function "_pearson_fit_ufunc" was compiled in object mode without forceobj=True.

File "C:\Users\ibrah\Anaconda3\lib\site-packages\climate_indices-1.0.9-py3.7.egg\climate_indices\compute.py", line 403: <source missing, REPL/exec in use?>

self.func_ir.loc)) C:\Users\ibrah\Anaconda3\lib\site-packages\numba\compiler.py:751: NumbaDeprecationWarning: Fall-back from the nopython compilation path to the object mode compilation path has been detected, this is deprecated behaviour.

For more information visit http://numba.pydata.org/numba-doc/latest/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit

File "C:\Users\ibrah\Anaconda3\lib\site-packages\climate_indices-1.0.9-py3.7.egg\climate_indices\compute.py", line 403: <source missing, REPL/exec in use?>

warnings.warn(errors.NumbaDeprecationWarning(msg, self.func_ir.loc)) C:\Users\ibrah\Anaconda3\lib\site-packages\climate_indices-1.0.9-py3.7.egg\climate_indices\compute.py:403: NumbaWarning: Compilation is falling back to object mode WITHOUT looplifting enabled because Function "_pearson_fit_ufunc" failed type inference due to: Untyped global name '_pearson3cdf': cannot determine Numba type of <class 'function'>

File "C:\Users\ibrah\Anaconda3\lib\site-packages\climate_indices-1.0.9-py3.7.egg\climate_indices\compute.py", line 446: <source missing, REPL/exec in use?>

2020-03-09 18:30:16 INFO Start time: 2020-03-09 18:30:16.676773 2020-03-09 18:30:16 ERROR Failed to complete Traceback (most recent call last): File "C:\Users\ibrah\Anaconda3\lib\site-packages\climate_indices-1.0.9-py3.7.egg\climate_indices__main__.py", line 1588, in main type=compute.Periodicity.from_string, File "C:\Users\ibrah\Anaconda3\lib\enum.py", line 349, in getattr raise AttributeError(name) from None AttributeError: from_string Traceback (most recent call last): File "C:\Users\ibrah\Anaconda3\Scripts\process_climate_indices-script.py", line 11, in load_entry_point('climate-indices==1.0.9', 'console_scripts', 'process_climate_indices')() File "C:\Users\ibrah\Anaconda3\lib\site-packages\climate_indices-1.0.9-py3.7.egg\climate_indices__main.py", line 1588, in main File "C:\Users\ibrah\Anaconda3\lib\enum.py", line 349, in getattr__ raise AttributeError(name) from None AttributeError: from_string

ibrah@DESKTOP-CBD1TAG MINGW64 /d/climate_indices/data (master) $

monocongo commented 4 years ago

My apologies, it is difficult for me to test further since I do not have a Windows machine available for testing.

At this point, my suggestion is to disable numba as this seems to be numba related (i.e. numba features that are supported on Linux aren't working as expected when running with numba on Windows). You can do this in two ways: 1) via setting the environment variable NUMBA_DISABLE_JIT which will disable numba altogether, or 2) removing the numba.jit decorator from the various Pearson fitting functions within compute.py. Option 1 is easier -- try setting the variable like so before running the SPI processing again:

$ export NUMBA_DISABLE_JIT=1
Romaysa commented 4 years ago

My apologies, it is difficult for me to test further since I do not have a Windows machine available for testing.

At this point, my suggestion is to disable numba as this seems to be numba related (i.e. numba features that are supported on Linux aren't working as expected when running with numba on Windows). You can do this in two ways: 1) via setting the environment variable NUMBA_DISABLE_JIT which will disable numba altogether, or 2) removing the numba.jit decorator from the various Pearson fitting functions within compute.py. Option 1 is easier -- try setting the variable like so before running the SPI processing again:

$ export NUMBA_DISABLE_JIT=1

Respected Sir, Shall i try it using Linux ubuntu i have a virtual machine. i am very confiused too i tried the first option to remove numba but still same error.

as its explained also in this page before runing the python SPI code i am running these test. https://www.drought.gov/drought/python-climate-indices

but its not working. what is your last advice for me?

Regards

monocongo commented 4 years ago

Do not follow the instructions on the drought.gov site as it is out of date (I no longer work there). Follow the instructions on the documentation from this repository. I am unclear on what to suggest further since I have verified that the code works as advertised as long as you fix your dataset as I've described. Please try again by doing the following:

  1. Create a new Anaconda environment, activate it
  2. Install the software with pip install climate-indices
  3. Disable numba JIT with export DISABLE_NUMBA_JIT=1
  4. Make sure that your dataset has a precipitation variable with valid units such as 'mm' (not 'mm/hr')

If you do the above then you should be OK for the SPI computation command.

Romaysa commented 4 years ago

My apologies, it is difficult for me to test further since I do not have a Windows machine available for testing. At this point, my suggestion is to disable numba as this seems to be numba related (i.e. numba features that are supported on Linux aren't working as expected when running with numba on Windows). You can do this in two ways: 1) via setting the environment variable NUMBA_DISABLE_JIT which will disable numba altogether, or 2) removing the numba.jit decorator from the various Pearson fitting functions within compute.py. Option 1 is easier -- try setting the variable like so before running the SPI processing again:

$ export NUMBA_DISABLE_JIT=1

Respected Sir, Shall i try it using Linux ubuntu i have a virtual machine. i am very confiused too i tried the first option to remove numba but still same error.

as its explained also in this page before runing the python SPI code i am running these test. https://www.drought.gov/drought/python-climate-indices

but its not working. what is your last advice for me?

Regards

Sir, process_climate_indices --index spi --periodicity monthly --netcdf_precip IMERGE_monthly_precip.nc --var_name_precip precipitation --output_file_base IMERG --scales 6 12 --calibration_start_year 2000 --calibration_end_year 2019 --multiprocessing all 2020-03-10 13:13:58 INFO Start time: 2020-03-10 13:13:58.940067 2020-03-10 13:13:59 INFO Computing 6-month SPI/Pearson 2020-03-10 13:14:17 INFO Computing 6-month SPI/Gamma 2020-03-10 13:14:29 INFO Computing 12-month SPI/Pearson 2020-03-10 13:14:46 INFO Computing 12-month SPI/Gamma 2020-03-10 13:14:59 INFO End time: 2020-03-10 13:14:59.329276 2020-03-10 13:14:59 INFO Elapsed time: 0:01:00.389209

Thank you very much sir, now it works well.

monocongo commented 4 years ago

Great news, I'm glad you have it working now. Kind regards...

bennyistanto commented 4 years ago

Hey @Romaysa, If you don't mind using geotiff, I'm more than happy to help with the code for converting netcdf to geotiff and calculating the 3 month SPI. The steps are:

  1. Convert netcdf to geotiff
  2. Read geotiff as numpy array
  3. Calculate SPI
  4. Save SPI array as geotiff

Hi @samburu As I am experiencing similar issue and found a discussion in this thread, is it possible to calculate the SPI with input from GeoTIFF files? If yes, would you mind to share the code and guideline?