romainsacchi
commented
2 years ago Hi, thanks for spotting this.
It seems the NDC scenario, being a bit old, does not have some of the necessary variables. I just push a commit that adds warning messages when the variables are missing.
/////////////////////// EXTRACTING IAM DATA ////////////////////////
The following variables cannot be found in the IAM file: ['SE|Gases|Non-Biomass']
The process continues with the remaining variables, but certain transformation functions may not work.
The following variables cannot be found in the IAM file: ['SE|Gases|Non-Biomass', 'Production|Industry|Cement', 'Production|Industry|Steel|Primary', 'Production|Industry|Steel|Secondary']
The process continues with the remaining variables, but certain transformation functions may not work.
No efficiency variables is given for the cement sector.
No efficiency variables is given for the primary steel sector.
No efficiency variables is given for the secondary steel sector.
Done!Consequently, certain transformation, like update_steel() or update_cement() won't work with this scenario.
A solution would be to ask REMIND's team to provide us with an updated NDC scenario.
marcosdbw
Lmachinferrero
Hello, I am running the most updated version of premise. When I try to create SSP2-NDC databases, an error appears when extracting the IAM Data (as shown below). I did not have any issues when creating databases for Base, PkBudg, etc using ecoinvent 3.6. Could you please help me to solve this? Thanks again!
//////////////////// EXTRACTING SOURCE DATABASE //////////////////// Done!
////////////////// IMPORTING DEFAULT INVENTORIES /////////////////// Done!
/////////////////////// EXTRACTING IAM DATA ////////////////////////
KeyError Traceback (most recent call last) Input In [17], in
3 from premise import *
4 import brightway2 as bw
----> 6 ndb = NewDatabase(
7 scenarios=[
8 {"model":"remind", "pathway":"SSP2-NDC", "year":2050}
9 ],
10 source_db="ecoinvent_3.6cutoff", # <-- name of the database in the BW2 project. Must be a string.
11 source_version="3.6", # <-- version of ecoinvent. Can be "3.5", "3.6", "3.7" or "3.7.1". Must be a string.
12 key='' # <-- decryption key
13 # to be requested from the library maintainers if you want ot use default scenarios included in
premise14 ) 15 16 ndb.update_all()File ~\Miniconda3\envs\bw2\lib\site-packages\premise\ecoinvent_modification.py:549, in NewDatabase.init(self, scenarios, source_version, source_type, key, source_db, source_file_path, additional_inventories, system_model, time_horizon, use_cached_inventories, use_cached_database) 546 print("\n/////////////////////// EXTRACTING IAM DATA ////////////////////////") 548 for scenario in self.scenarios: --> 549 scenario["external data"] = IAMDataCollection( 550 model=scenario["model"], 551 pathway=scenario["pathway"], 552 year=scenario["year"], 553 filepath_iam_files=scenario["filepath"], 554 key=key, 555 system_model=self.system_model, 556 time_horizon=self.time_horizon, 557 ) 558 scenario["database"] = copy.deepcopy(self.database) 560 print("Done!")
File ~\Miniconda3\envs\bw2\lib\site-packages\premise\data_collection.py:173, in IAMDataCollection.init(self, model, pathway, year, filepath_iam_files, key, system_model, time_horizon) 170 self.gnr_data = get_gnr_data() 172 self.electricity_markets = self.get_iam_electricity_markets(data=data) --> 173 self.fuel_markets = self.get_iam_fuel_markets(data=data) 175 prod_vars = self.__get_iam_variable_labels(IAM_ELEC_VARS, key="iam_aliases") 176 prod_vars.update( 177 self.__get_iam_variable_labels(IAM_FUELS_VARS, key="iam_aliases") 178 )
File ~\Miniconda3\envs\bw2\lib\site-packages\premise\data_collection.py:948, in IAMDataCollection.__get_iam_fuel_markets(self, data) 939 raise KeyError( 940 f"{self.year} is outside of the boundaries " 941 f"of the IAM file: {data.year.values.min()}-{data.year.values.max()}" 942 ) 944 # Finally, if the specified year falls in between two periods provided by the IAM 945 # sometimes, the World region is either neglected 946 # or wrongly evaluated so we fix that here --> 948 data.loc[dict(region="World", variables=list_technologies)] = data.loc[ 949 dict( 950 region=[r for r in data.coords["region"].values if r != "World"], 951 variables=list_technologies, 952 ) 953 ].sum(dim="region") 955 # Interpolation between two periods 956 data_to_return = data.loc[:, list_technologies, :]
File ~\Miniconda3\envs\bw2\lib\site-packages\xarray\core\dataarray.py:198, in _LocIndexer.getitem(self, key) 196 labels = indexing.expanded_indexer(key, self.data_array.ndim) 197 key = dict(zip(self.data_array.dims, labels)) --> 198 return self.data_array.sel(key)
File ~\Miniconda3\envs\bw2\lib\site-packages\xarray\core\dataarray.py:1328, in DataArray.sel(self, indexers, method, tolerance, drop, indexers_kwargs) 1219 def sel( 1220 self, 1221 indexers: Mapping[Any, Any] = None, (...) 1225 indexers_kwargs: Any, 1226 ) -> DataArray: 1227 """Return a new DataArray whose data is given by selecting index 1228 labels along the specified dimension(s). 1229 (...) 1326 Dimensions without coordinates: points 1327 """ -> 1328 ds = self._to_temp_dataset().sel( 1329 indexers=indexers, 1330 drop=drop, 1331 method=method, 1332 tolerance=tolerance, 1333 **indexers_kwargs, 1334 ) 1335 return self._from_temp_dataset(ds)
File ~\Miniconda3\envs\bw2\lib\site-packages\xarray\core\dataset.py:2500, in Dataset.sel(self, indexers, method, tolerance, drop, **indexers_kwargs) 2439 """Returns a new dataset with each array indexed by tick labels 2440 along the specified dimension(s). 2441 (...) 2497 DataArray.sel 2498 """ 2499 indexers = either_dict_or_kwargs(indexers, indexers_kwargs, "sel") -> 2500 pos_indexers, new_indexes = remap_label_indexers( 2501 self, indexers=indexers, method=method, tolerance=tolerance 2502 ) 2503 # TODO: benbovy - flexible indexes: also use variables returned by Index.query 2504 # (temporary dirty fix). 2505 new_indexes = {k: v[0] for k, v in new_indexes.items()}
File ~\Miniconda3\envs\bw2\lib\site-packages\xarray\core\coordinates.py:421, in remap_label_indexers(obj, indexers, method, tolerance, **indexers_kwargs) 414 indexers = either_dict_or_kwargs(indexers, indexers_kwargs, "remap_label_indexers") 416 v_indexers = { 417 k: v.variable.data if isinstance(v, DataArray) else v 418 for k, v in indexers.items() 419 } --> 421 pos_indexers, new_indexes = indexing.remap_label_indexers( 422 obj, v_indexers, method=method, tolerance=tolerance 423 ) 424 # attach indexer's coordinate to pos_indexers 425 for k, v in indexers.items():
File ~\Miniconda3\envs\bw2\lib\site-packages\xarray\core\indexing.py:121, in remap_label_indexers(data_obj, indexers, method, tolerance) 119 for dim, index in indexes.items(): 120 labels = grouped_indexers[dim] --> 121 idxr, new_idx = index.query(labels, method=method, tolerance=tolerance) 122 pos_indexers[dim] = idxr 123 if new_idx is not None:
File ~\Miniconda3\envs\bw2\lib\site-packages\xarray\core\indexes.py:247, in PandasIndex.query(self, labels, method, tolerance) 245 indexer = get_indexer_nd(self.index, label, method, tolerance) 246 if np.any(indexer < 0): --> 247 raise KeyError(f"not all values found in index {coord_name!r}") 249 return indexer, None
KeyError: "not all values found in index 'variables'"