search

jrising / prospectus-tools

Tools for processing results of the Climate Prospectus
MIT License
2 stars 7 forks source link

Error while extracting energy impact with full uncertainty #41

Open ruixue-li opened 3 years ago

ruixue-li commented 3 years ago

Hi @jrising and @brews ,

I'm trying to extract some energy impact with full uncertainty with this config, and this command:

python -u quantiles.py /home/liruixue/repos/energy-code-release-2020/projection_inputs/configs/GMFD/TINV_clim/break2_Exclude/semi-parametric/Extraction_Configs/sacagawea/impactpc//full/aggregated/median/energy-extract-impactpc-aggregated-median_OTHERIND_electricity.yml --only-iam=low --suffix=_test_impactpc_median_fulluncertainty_low_fulladapt-aggregated FD_FGLS_inter_OTHERIND_electricity_TINV_clim-aggregated -FD_FGLS_inter_OTHERIND_electricity_TINV_clim-histclim-aggregated

But I'm getting the following error at the end:

...
/shares/gcp/outputs/energy_pixel_interaction/impacts-blueghost/median_OTHERIND_electricity_TINV_clim_GMFD_dm/median/rcp85/MIROC5/high/SSP2 not low
/shares/gcp/outputs/energy_pixel_interaction/impacts-blueghost/median_OTHERIND_electricity_TINV_clim_GMFD_dm/median/rcp85/CanESM2/low/SSP3
/shares/gcp/outputs/energy_pixel_interaction/impacts-blueghost/median_OTHERIND_electricity_TINV_clim_GMFD_dm/median/rcp85/CanESM2/high/SSP4 not low
/shares/gcp/outputs/energy_pixel_interaction/impacts-blueghost/median_OTHERIND_electricity_TINV_clim_GMFD_dm/median/rcp85/CanESM2/high/SSP5 not low
/shares/gcp/outputs/energy_pixel_interaction/impacts-blueghost/median_OTHERIND_electricity_TINV_clim_GMFD_dm/median/rcp85/CanESM2/high/SSP3 not low
/shares/gcp/outputs/energy_pixel_interaction/impacts-blueghost/median_OTHERIND_electricity_TINV_clim_GMFD_dm/median/rcp85/CanESM2/high/SSP2 not low
Observations: 15546
Creating file: ('SSP4', 'rcp85')
('SSP4', 'rcp85') is not in delta method output. Skipping model specification...
Creating file: ('SSP4', 'rcp45')
('SSP4', 'rcp45') is not in delta method output. Skipping model specification...
Creating file: ('SSP5', 'rcp85')
('SSP5', 'rcp85') is not in delta method output. Skipping model specification...
Creating file: ('SSP1', 'rcp45')
('SSP1', 'rcp45') is not in delta method output. Skipping model specification...
Creating file: ('SSP2', 'rcp85')
('SSP2', 'rcp85') is not in delta method output. Skipping model specification...
Creating file: ('SSP2', 'rcp45')
('SSP2', 'rcp45') is not in delta method output. Skipping model specification...
Creating file: ('SSP3', 'rcp85')
Outputing row: ('all', 1981)
/home/liruixue/miniconda3/envs/risingverse-py27/lib/python2.7/site-packages/scipy/stats/_distn_infrastructure.py:1913: RuntimeWarning: invalid value encountered in greater
  cond0 = self._argcheck(*args) & (scale > 0) & (loc == loc)
/home/liruixue/miniconda3/envs/risingverse-py27/lib/python2.7/site-packages/scipy/stats/_distn_infrastructure.py:1746: RuntimeWarning: invalid value encountered in greater
  cond0 = self._argcheck(*args) & (scale > 0)
/home/liruixue/miniconda3/envs/risingverse-py27/lib/python2.7/site-packages/scipy/stats/_distn_infrastructure.py:877: RuntimeWarning: invalid value encountered in greater
  return (self.a < x) & (x < self.b)
/home/liruixue/miniconda3/envs/risingverse-py27/lib/python2.7/site-packages/scipy/stats/_distn_infrastructure.py:877: RuntimeWarning: invalid value encountered in less
  return (self.a < x) & (x < self.b)
/home/liruixue/miniconda3/envs/risingverse-py27/lib/python2.7/site-packages/scipy/stats/_distn_infrastructure.py:1747: RuntimeWarning: invalid value encountered in greater
  cond1 = self._open_support_mask(x) & (scale > 0)
/home/liruixue/miniconda3/envs/risingverse-py27/lib/python2.7/site-packages/scipy/stats/_distn_infrastructure.py:1748: RuntimeWarning: invalid value encountered in greater_equal
  cond2 = (x >= self.b) & cond0
Traceback (most recent call last):
  File "quantiles.py", line 114, in <module>
    writer.writerow(myrowstuff + list(distribution.inverse(encoded_evalqvals)))
  File "/home/liruixue/repos/prospectus-tools/gcp/extract/lib/weights_vcv.py", line 54, in inverse
    roots.append(brentq(func, left, right))
  File "/home/liruixue/miniconda3/envs/risingverse-py27/lib/python2.7/site-packages/scipy/optimize/zeros.py", line 754, in brentq
    r = _zeros._brentq(f, a, b, xtol, rtol, maxiter, args, full_output, disp)
RuntimeError: Failed to converge after 100 iterations.

This error does not occur if I limit to one or a set of regions, or if I extract the non-aggregated files.

I'm wondering if any of you have any idea what's going on? I could work around this and get what I need by specifying regions, but I'd just like to make sure that I'm not doing anything that I shouldn't be doing, or if there's an obvious and easy fix for it.

Thank you very much!

jrising commented 3 years ago

It appears that you're getting a bunch of NAs for some of the regions. Checking into it.

jrising commented 3 years ago

It looks like there are missing values in the following regions: TWN.4 TWN.3 TWN.1 TWN.2 NZL.10.42 HMD CHN.21.226 CHN.21 ATF ZAF.9 SGS CL- BRA.8.837 BRA.8.836 AUS.1 AUS.3 ESP.6.27.191 ESP.6.27.192 SP- IOT BVT ATA

These are generally very small regions. I have a fix where the code just warns you that there are missing values and so it has to skip the region, but it might be worthwhile to find out why there are these values.

ruixue-li commented 3 years ago

Hi @jrising ,

I see, thanks! If it's not too much trouble, do you mind putting in that fix first, so that it doesn't hold up other stuff while I look into why those regions are missing?

jrising commented 3 years ago

Sure. I've just committed it to master. Here are the changes: https://github.com/jrising/prospectus-tools/commit/a22539b66278b853159fe0343c38a24fecee1fc5