Bugs when importing a SDF

[bsteubing]

Updating AB

• [X] Yes, I have updated AB and still experience this issue

What happened?

There seem to be several bugs when importing an SDF (that is not 100% correct):

1) when the database name does not match (and the user supplies the correct name via a dialogue in the AB): i.e. superstructure name in AB is "Ax" and superstructure name in SDF is "Ay" (it is the correct superstructure, just called differenty)

We get the following error (and this potentially millions of times, I think as many rows * scenario columns):

  ds_.loc[col] = (key['database'][0], key['code'][0]) if j == 0 else key['database'][0]
0.17224634282861595C:\Users\steub\miniforge3\envs\abm\lib\site-packages\activity_browser\bwutils\superstructure\dataframe.py:142: FutureWarning: Series.__getitem__ treating keys as positions is deprecated. In a future version, integer keys will always be treated as labels (consistent with DataFrame behavior). To access a value by position, use `ser.iloc[pos]`

**2) if the keys are missing:** (resolved in #1069)

Traceback (most recent call last): File "C:\Users\steub\Bernie\surfdrive\Leiden\python\GitHub\LCA-ActivityBrowser\activity-browser\activity_browser\bwutils\superstructure\utils.py", line 69, in wrapper result = func(*args) File "C:\Users\steub\Bernie\surfdrive\Leiden\python\GitHub\LCA-ActivityBrowser\activity-browser\activity_browser\layouts\tabs\LCA_setup.py", line 530, in load_action self.sync_superstructure(df) File "C:\Users\steub\Bernie\surfdrive\Leiden\python\GitHub\LCA-ActivityBrowser\activity-browser\activity_browser\bwutils\superstructure\utils.py", line 69, in wrapper result = func(*args) File "C:\Users\steub\Bernie\surfdrive\Leiden\python\GitHub\LCA-ActivityBrowser\activity-browser\activity_browser\layouts\tabs\LCA_setup.py", line 591, in sync_superstructure df = SuperstructureManager.fill_empty_process_keys_in_exchanges(df) File "C:\Users\steub\Bernie\surfdrive\Leiden\python\GitHub\LCA-ActivityBrowser\activity-browser\activity_browser\bwutils\superstructure\utils.py", line 69, in wrapper result = func(*args) File "C:\Users\steub\Bernie\surfdrive\Leiden\python\GitHub\LCA-ActivityBrowser\activity-browser\activity_browser\bwutils\superstructure\manager.py", line 376, in fill_empty_process_keys_in_exchanges df = fill_df_keys_with_fields(df) File "C:\Users\steub\Bernie\surfdrive\Leiden\python\GitHub\LCA-ActivityBrowser\activity-browser\activity_browser\bwutils\superstructure\activities.py", line 137, in fill_df_keys_with_fields matches = get_relevant_flows(df, "from") File "C:\Users\steub\Bernie\surfdrive\Leiden\python\GitHub\LCA-ActivityBrowser\activity-browser\activity_browser\bwutils\superstructure\activities.py", line 116, in get_relevant_flows names, categories, dbs = sub.iloc[:, 0:3].apply(set, axis=0) File "C:\Users\steub\miniconda3\envs\ab\lib\site-packages\pandas\core\frame.py", line 10037, in apply return op.apply().__finalize__(self, method="apply") File "C:\Users\steub\miniconda3\envs\ab\lib\site-packages\pandas\core\apply.py", line 837, in apply return self.apply_standard() File "C:\Users\steub\miniconda3\envs\ab\lib\site-packages\pandas\core\apply.py", line 966, in apply_standard return self.wrap_results(results, res_index) File "C:\Users\steub\miniconda3\envs\ab\lib\site-packages\pandas\core\apply.py", line 992, in wrap_results return self.wrap_results_for_axis(results, res_index) File "C:\Users\steub\miniconda3\envs\ab\lib\site-packages\pandas\core\apply.py", line 1055, in wrap_results_for_axis result = self.obj._constructor(data=results) File "C:\Users\steub\miniconda3\envs\ab\lib\site-packages\pandas\core\frame.py", line 736, in __init__ mgr = dict_to_mgr(data, index, columns, dtype=dtype, copy=copy, typ=manager) File "C:\Users\steub\miniconda3\envs\ab\lib\site-packages\pandas\core\internals\construction.py", line 503, in dict_to_mgr return arrays_to_mgr(arrays, columns, index, dtype=dtype, typ=typ, consolidate=copy) File "C:\Users\steub\miniconda3\envs\ab\lib\site-packages\pandas\core\internals\construction.py", line 119, in arrays_to_mgr arrays, refs = _homogenize(arrays, index, dtype) File "C:\Users\steub\miniconda3\envs\ab\lib\site-packages\pandas\core\internals\construction.py", line 629, in _homogenize val = sanitize_array(val, index, dtype=dtype, copy=False) File "C:\Users\steub\miniconda3\envs\ab\lib\site-packages\pandas\core\construction.py", line 619, in sanitize_array _sanitize_non_ordered(data) File "C:\Users\steub\miniconda3\envs\ab\lib\site-packages\pandas\core\construction.py", line 670, in _sanitize_non_ordered raise TypeError(f"'{type(data).__name__}' type is unordered") TypeError: 'set' type is unordered

Relevant errors

No response

Operating system

Windows 10

Conda environment

No response

[Zoophobus]

@bsteubing, the 1st point needs some thought before we tackle it. Maybe the dataframes need a slightly different structure.

For the 2nd issue there is already a pull request to resolve this, it's related to issue #1062

[bsteubing]

Indeed, 2) is solved