search

EnergySystemsModellingLab / MUSE_OS

Welcome to the MUSE-OS repository
https://muse-os.readthedocs.io/en/latest/
GNU General Public License v3.0
22 stars 9 forks source link

Error message caused by demand sharing with multiple agents [BUG] #252

Closed LennartMorlock closed 4 months ago

LennartMorlock commented 6 months ago

Describe the bug

When running MUSE with multiple agents with different objectives, search rules or decision methods, the error message below often occurs. This also relates to a post in the MUSE Google group.

Traceback (most recent call last): File "C:\Users\lenna\anaconda3\envs\muse_env\lib\runpy.py", line 197, in _run_module_as_main return _run_code(code, main_globals, None, File "C:\Users\lenna\anaconda3\envs\muse_env\lib\runpy.py", line 87, in _run_code exec(code, run_globals) File "C:\Users\lenna\anaconda3\envs\muse_env\Scripts\muse.exe__main.py", line 7, in File "C:\Users\lenna\MUSE_test2\MUSE_OS\src\muse__main__.py", line 60, in run muse_main(args.settings, args.model, args.copy) File "C:\Users\lenna\MUSE_test2\MUSE_OS\src\muse__main__.py", line 55, in muse_main MCA.factory(settings).run() File "C:\Users\lenna\MUSE_test2\MUSEOS\src\muse\mca.py", line 346, in run , new_market, self.sectors = self.find_equilibrium(new_market) File "C:\Users\lenna\MUSE_test2\MUSE_OS\src\muse\mca.py", line 209, in find_equilibrium return find_equilibrium( File "C:\Users\lenna\MUSE_test2\MUSE_OS\src\muse\mca.py", line 561, in find_equilibrium market, equilibrium_sectors = single_year_iteration(market, sectors) File "C:\Users\lenna\MUSE_test2\MUSE_OS\src\muse\mca.py", line 468, in single_year_iteration sector_market = sector.next( File "C:\Users\lenna\MUSE_test2\MUSE_OS\src\muse\sectors\sector.py", line 236, in next subsector.invest( File "C:\Users\lenna\MUSE_test2\MUSE_OS\src\muse\sectors\subsector.py", line 74, in invest lp_problem = self.aggregate_lp( File "C:\Users\lenna\MUSE_test2\MUSE_OS\src\muse\sectors\subsector.py", line 147, in aggregate_lp result = agent.next( File "C:\Users\lenna\MUSE_test2\MUSE_OS\src\muse\agents\agent.py", line 421, in next investments = self.invest( File "C:\Users\lenna\MUSE_test2\MUSE_OS\src\muse\investments.py", line 168, in compute_investment return investment( File "C:\Users\lenna\MUSE_test2\MUSE_OS\src\muse\registration.py", line 172, in decorated result = inner_decorated(args, kwargs) File "C:\Users\lenna\MUSE_test2\MUSE_OS\src\muse\investments.py", line 99, in decorated result = function(costs, search_space, technologies, constraints, kwargs) File "C:\Users\lenna\MUSE_test2\MUSE_OS\src\muse\investments.py", line 286, in adhoc_match_demand production = demand_matching( File "C:\Users\lenna\MUSE_test2\MUSE_OS\src\muse\demand_matching.py", line 249, in demand_matching result = demand_matching( # type: ignore File "C:\Users\lenna\MUSE_test2\MUSE_OS\src\muse\demand_matching.py", line 255, in demand_matching return _demand_matching_impl(demand, cost, constraints) File "C:\Users\lenna\MUSE_test2\MUSE_OS\src\muse\demand_matching.py", line 297, in _demand_matchingimpl for , same_cost in data.groupby("cost") if cost.dims else [(cost, data)]: File "C:\Users\lenna\anaconda3\envs\muse_env\lib\site-packages\xarray\core\common.py", line 721, in groupby return self._groupby_cls( File "C:\Users\lenna\anaconda3\envs\muse_env\lib\site-packages\xarray\core\groupby.py", line 385, in init__ raise ValueError( ValueError: Failed to group data. Are you grouping by a variable that is all NaN?

To Reproduce

Running MUSE with the attached input files causes this error. However, this error has occurred in many different setups.

base_model_nopowse_errorsearch (3).zip

Probable cause

I traced the error to the demand_share.py file _inner_split() function line 479, where a variable called "unassigned" is calculated. In the cases where the error message occurs, the term "len(cast(xr.DataArray, sum(shares.values())).asset)" becomes zero, which leads to unassigned getting some "inf" values, which I think cause the NaN further down the line.

I think due to the functionality of the sum() function, the term becomes zero if the arrays referenced by shares.values() don't share a common asset type. This is the case if all agents don't share at least one common technology, which is why it only occurs when adding agents with different decision-making processes. Similarly, adding existing capacity solves this as the initial capacity is distributed between all agents. Thus, if some small amount stays until the end of the model run, all agents will always have at least this technology as a common asset type.

Possible Solution

Replacing the term sum(shares.values()) by xr.concat(shares.values(), dim='concat_dim').sum('concat_dim') for both occurrences in the _inner_split() function removes that error. The results look alright, too, on first inspection.

However, I am unsure what the purpose of the unassigned variable is and, thus, if the proposed change makes sense from a modelling perspective.

Context

Please, complete the following to better understand the system you are using to run MUSE.

sgiarols commented 6 months ago

Thanks. I can deal with this coming week

sgiarols commented 6 months ago

Just started to look at this, first thing that appears is that for many sectors, the "Quantity" attribute is quite disassociated from the technodata. This results in a considerable over-capacity.

LennartMorlock commented 4 months ago

simple_model_error.zip

Sorry for not responding earlier. I have now attached a much-simplified model that should not have this disassociation and still leads to the same error.

Best Lennart