`xarray` interp issue

[romainsacchi]

After pip install premise==1.5.0-b3 I tried to run a premise script for a ecoinvent 3.9.1 loaded on bw2 but I got an error

/////////////////////// EXTRACTING IAM DATA ////////////////////////

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
~\AppData\Local\Temp\ipykernel_21116\732254875.py in <module>
----> 1 ndb = NewDatabase(
      2             scenarios=[
      3                 {"model":"image", "pathway":"SSP2-RCP19", "year":2030},
      4                 {"model":"image", "pathway":"SSP2-RCP19", "year":2040},
      5                 {"model":"image", "pathway":"SSP2-RCP19", "year":2050},

~\anaconda3\lib\site-packages\premise\ecoinvent_modification.py in __init__(self, scenarios, source_version, source_type, key, source_db, source_file_path, additional_inventories, system_model, system_args, use_cached_inventories, use_cached_database, external_scenarios, quiet, keep_uncertainty_data, gains_scenario)
    518 
    519         for scenario in self.scenarios:
--> 520             data = IAMDataCollection(
    521                 model=scenario["model"],
    522                 pathway=scenario["pathway"],

~\anaconda3\lib\site-packages\premise\data_collection.py in __init__(self, model, pathway, year, filepath_iam_files, key, external_scenarios, system_model, system_model_args, gains_scenario)
    320         self.system_model = system_model
    321 
--> 322         self.gains_data_EU = get_gains_EU_data()
    323         self.gains_data_IAM = get_gains_IAM_data(
    324             self.model, gains_scenario=gains_scenario

~\anaconda3\lib\site-packages\premise\data_collection.py in get_gains_EU_data()
    160     )
    161 
--> 162     array = array.interpolate_na(dim="year", method="nearest", fill_value="extrapolate")
    163     array = array.bfill(dim="year")
    164     array = array.ffill(dim="year")

~\anaconda3\lib\site-packages\xarray\core\dataarray.py in interpolate_na(self, dim, method, limit, use_coordinate, max_gap, keep_attrs, **kwargs)
   2570         from .missing import interp_na
   2571 
-> 2572         return interp_na(
   2573             self,
   2574             dim=dim,

~\anaconda3\lib\site-packages\xarray\core\missing.py in interp_na(self, dim, use_coordinate, method, limit, max_gap, keep_attrs, **kwargs)
    346         warnings.filterwarnings("ignore", "overflow", RuntimeWarning)
    347         warnings.filterwarnings("ignore", "invalid value", RuntimeWarning)
--> 348         arr = apply_ufunc(
    349             interpolator,
    350             self,

~\anaconda3\lib\site-packages\xarray\core\computation.py in apply_ufunc(func, input_core_dims, output_core_dims, exclude_dims, vectorize, join, dataset_join, dataset_fill_value, keep_attrs, kwargs, dask, output_dtypes, output_sizes, meta, dask_gufunc_kwargs, *args)
   1163     # feed DataArray apply_variable_ufunc through apply_dataarray_vfunc
   1164     elif any(isinstance(a, DataArray) for a in args):
-> 1165         return apply_dataarray_vfunc(
   1166             variables_vfunc,
   1167             *args,

~\anaconda3\lib\site-packages\xarray\core\computation.py in apply_dataarray_vfunc(func, signature, join, exclude_dims, keep_attrs, *args)
    288 
    289     data_vars = [getattr(a, "variable", a) for a in args]
--> 290     result_var = func(*data_vars)
    291 
    292     if signature.num_outputs > 1:

~\anaconda3\lib\site-packages\xarray\core\computation.py in apply_variable_ufunc(func, signature, exclude_dims, dask, output_dtypes, vectorize, keep_attrs, dask_gufunc_kwargs, *args)
    731             )
    732 
--> 733     result_data = func(*input_data)
    734 
    735     if signature.num_outputs == 1:

~\anaconda3\lib\site-packages\numpy\lib\function_base.py in __call__(self, *args, **kwargs)
   2161             vargs.extend([kwargs[_n] for _n in names])
   2162 
-> 2163         return self._vectorize_call(func=func, args=vargs)
   2164 
   2165     def _get_ufunc_and_otypes(self, func, args):

~\anaconda3\lib\site-packages\numpy\lib\function_base.py in _vectorize_call(self, func, args)
   2235         """Vectorized call to `func` over positional `args`."""
   2236         if self.signature is not None:
-> 2237             res = self._vectorize_call_with_signature(func, args)
   2238         elif not args:
   2239             res = func()

~\anaconda3\lib\site-packages\numpy\lib\function_base.py in _vectorize_call_with_signature(self, func, args)
   2275 
   2276         for index in np.ndindex(*broadcast_shape):
-> 2277             results = func(*(arg[index] for arg in args))
   2278 
   2279             n_results = len(results) if isinstance(results, tuple) else 1

~\anaconda3\lib\site-packages\xarray\core\missing.py in func_interpolate_na(interpolator, y, x, **kwargs)
    387         return y
    388 
--> 389     f = interpolator(x[nonans], y[nonans], **kwargs)
    390     out[nans] = f(x[nans])
    391     return out

~\anaconda3\lib\site-packages\xarray\core\missing.py in __init__(self, xi, yi, method, fill_value, assume_sorted, copy, bounds_error, order, **kwargs)
    149             fill_value = np.nan
    150 
--> 151         self.f = interp1d(
    152             xi,
    153             yi,

~\anaconda3\lib\site-packages\scipy\interpolate\_interpolate.py in __init__(***failed resolving arguments***)
    577 
    578         if len(self.x) < minval:
--> 579             raise ValueError("x and y arrays must have at "
    580                              "least %d entries" % minval)
    581 

ValueError: x and y arrays must have at least 2 entries

[romainsacchi]

Things work well on my end, so let's try to check a few basic things.

What is the version of xarray installed in your environment? Can you confirm the presence of a CSV file under premise/premise/data/GAINS_emission_factors/GAINS_emission_factors_EU.csv? If so, can you confirm the values are separated by a comma (",")?

Does it show exactly this -->

Region,Sector,Activity,value,year,substance AL,CON_COMB,GAS,0.019,2005,CO AL,CON_COMB,HF,0.04,2005,CO

[mintony89]

Many thanks Romain.

xarray: 0.20.1 I can confirm I have that .csv file in that location, however values are separated by a dot "." It refers as CSV type of files but it shows values separated by column without commas.

[romainsacchi]

That's surprising. Originally, teh data is comma-separated -- see. OK, I'll find a way to automatically detect the value separator.

[aaradhya99]

Dear Romain, I just checked the csv file and I have it separated by commas. But, still the similar error persists.

[mintony89]

Me too - issue persists even changing the csv file to separate commas.

[romainsacchi]

Hi @mintony89 , @aaradhya99 made a clean installation of premise in a dedicated and separate conda environment, and this seemed to solve the issue -- he has another one, though, but unrelated.

[romainsacchi]

Hi @mintony89 can you try upgrading xarray to 2023.4.2? You can directly do pip install xarray==2023.4.2

[romainsacchi]

Hi @mintony89 @aaradhya99 ,

could you let me know if installing v.1.5.0-b4 in a clean new environment solve all the issues?

[aaradhya99]

Dear @romainsacchi Sure, Let me go through the process, and I shall update you soon.