Getting NA when extracting energy quantiles: "Cannot find the location of 0.500000 for the following means and std. devs"

[ruixue-li]

Hi @jrising ,

I'm getting a NA for Florida (USA.10), year 2020 when extracting the q50 quantile of total damages. The mean value of the same region and year also looks wrong (a sudden large negative value in a time series of large positive values).

There's nothing missing in our projection output, and I found this error message in the log file. I'm wondering if you have any thoughts on what may have caused this? Thank you so much.

Cannot find the location of 0.500000 for the following means and std. devs:
[156352.69, 220194.66, -135093.42, 51638.164, 38788.71, -247344.86, 138134.25, -110481.79, -111354.6, 19424.445, 86202.59, 49737.1, 322012.53, -149590.02, 30932.445, 51013.586, -201455.69, -34580.297, 239524.81, -159094.3, 63949.312, -21903.66, -103427.54, 75123.09, 179677.38, -155073.27, 19809.742, 18409.633, -181643.39, -108001.23, 246619.53, -7087.758, -293796.3]
[145443.09140464 116285.63551667 194286.41702908  38250.07524274
 139477.616702   202931.91396146  46952.36697973 219148.73996463
 145352.00169308 112583.030585    74008.17514476  33283.37914394
 128034.59104713 192243.22625885  39805.58539383  36240.15215872
 316390.31523151 137298.49178622 121194.68042582 173380.38772798
  99666.97483252  14926.8093978  167903.98701365 234969.86082831
 167596.09847793 181376.3249282  114558.75803963 109542.92729747
  69749.39618954 154912.59264882  97062.20301687 120904.03468124
 142090.07889108]

Here's my extraction command for your reference:

nohup python -u quantiles.py /home/liruixue/repos/energy-code-release-2020/projection_inputs/configs/GMFD/TINV_clim/break2_Exclude/semi-parametric/Extraction_Configs/sacagawea/damage/price014/full/aggregated/median/energy-extract-damage-aggregated-price014-median_OTHERIND_total_energy.yml --only-iam=low --only-ssp=SSP3 --only-rcp=rcp85 --regions=[USA.13,USA.27,USA.35,USA.23,USA.48,USA.50,USA.20,USA.42,USA.51,USA.33,USA.30,USA.16,USA.29,USA.6,USA.15,USA.36,USA.28,USA.39,USA.1,USA.11,USA.37,USA.32,USA.41,USA.44,USA.4,USA.19,USA.24,USA.38,USA.43,USA.34,USA.18,USA.47,USA.14,USA.31,USA.26,USA.21,USA.45,USA.2,USA.49,USA.22,USA.5,USA.46,USA.17,USA.25,USA.7,USA.40,USA.9,USA.8,USA.3,USA.10,USA.12] --suffix=_states_damage-price014_median_fulluncertainty_low_fulladapt-aggregated FD_FGLS_inter_OTHERIND_electricity_TINV_clim-price014-aggregated -FD_FGLS_inter_OTHERIND_electricity_TINV_clim-histclim-price014-aggregated FD_FGLS_inter_OTHERIND_other_energy_TINV_clim-price014-aggregated -FD_FGLS_inter_OTHERIND_other_energy_TINV_clim-histclim-price014-aggregated >> /home/liruixue/extraction_shell_logs/log_states_damage-price014_median_fulluncertainty_low_fulladapt-aggregated_OTHERIND_total_energy.txt

And here's the config referenced in the command above:

results-root:
  FD_FGLS_inter_OTHERIND_other_energy_TINV_clim.*: /shares/gcp/outputs/energy_pixel_interaction/impacts-blueghost/median_OTHERIND_other_energy_TINV_clim_GMFD
  FD_FGLS_inter_OTHERIND_electricity_TINV_clim.*: /shares/gcp/outputs/energy_pixel_interaction/impacts-blueghost/median_OTHERIND_electricity_TINV_clim_GMFD
deltamethod:
  FD_FGLS_inter_OTHERIND_other_energy_TINV_clim.*: /shares/gcp/outputs/energy_pixel_interaction/impacts-blueghost/median_OTHERIND_other_energy_TINV_clim_GMFD_dm
  FD_FGLS_inter_OTHERIND_electricity_TINV_clim.*: /shares/gcp/outputs/energy_pixel_interaction/impacts-blueghost/median_OTHERIND_electricity_TINV_clim_GMFD_dm
multiimpact_vcv: /home/liruixue/repos/energy-code-release-2020/projection_inputs/csvv/TINV_clim/FD_FGLS_inter_OTHERIND_TINV_clim.csv
output-dir: /shares/gcp/social/parameters/energy_pixel_interaction/extraction/multi-models/rationalized_code/break2_Exclude_all-issues_semi-parametric/TINV_clim_GMFD/total_energy
file-organize: [ssp, rcp]
do-montecarlo: no
only-models: all
only-iam: null
column: rebased
evalqvals: ['mean', .5, 0.05, 0.17, 0.83, 0.95, 0.10, 0.90, 0.75, 0.25]
output-format: edfcsv

[jrising]

I started looking into this. It works fine when run with just the data reported-- means and std. dev.s, and applying a weight of 1 to all models. It might be a problem with the weight values, but I need to check more.

[jrising]

@ruixue-li Where can I get this file: /home/liruixue/repos/energy-code-release-2020/projection_inputs/csvv/TINV_clim/FD_FGLS_inter_OTHERIND_TINV_clim.csv? What's the repo, or can we move it to a shared spot?

[ruixue-li]

Hi @jrising ,

It's here , and I've moved it to

/mnt/CIL_energy/code_release_data_pixel_interaction/miscellaneous/FD_FGLS_inter_OTHERIND_TINV_clim.csv

[jrising]

@ruixue-li Okay, I figured it out. The wide range of values ended up just making it so that the fixed point wasn't found within 100 iterations, which was the default limit. I just increased it to 200 iterations, and it seems to work fine now!

[ruixue-li]

Thank you so much @jrising !