irena-flextool / flextool

IRENA FlexTool is an energy and power systems model for understanding the role of variable power generation in future energy systems.
https://www.irena.org/energytransition/Energy-System-Models-and-Data/IRENA-FlexTool
Other
22 stars 5 forks source link

Importers for connections do not work anymore #216

Closed e-zaline closed 1 month ago

e-zaline commented 1 month ago

This importer works:

image

This importer does not work (it does not import anything):

image

Some times ago it used to work. I don't know if it is due to FlexTool or SpineToolbox, but this is a very blocking issue. It happens with all parameters related to connections, and only to those related to connections (no problem with nodes).

soininen commented 1 month ago

Do you get any import errors in Event log? Any tracebacks in console?

e-zaline commented 1 month ago

That might be a SpineToolbox issue, because when checking the CSV p_process, I do see the parameters:

image

but not in SpineToolbox:

image
soininen commented 1 month ago

Is the Database editor open when you execute the importer? Is this a refreshing issue?

e-zaline commented 1 month ago

No it's not open. I have tried closing and reopening Spinetoolbox, it still does not appear.

e-zaline commented 1 month ago

When I open the DB editor, I have this message in the Anaconda console, not sure if it's related:

(flextool) C:\Users\pc>python -m spinetoolbox
Traceback (most recent call last):
  File "C:\Users\pc\Spine-Toolbox\spinetoolbox\helpers.py", line 136, in new_function
    return func(*args, **kwargs)
  File "C:\Users\pc\Spine-Toolbox\spinetoolbox\fetch_parent.py", line 99, in _apply_pending_changes
    last_handler({db_map: items})
  File "C:\Users\pc\Spine-Toolbox\spinetoolbox\fetch_parent.py", line 322, in handle_items_added
    self._handle_items_added(db_map_data)
  File "C:\Users\pc\Spine-Toolbox\spinetoolbox\spine_db_editor\mvcmodels\compound_models.py", line 343, in handle_items_added
    self._add_items(db_map, entity_class_id, ids_committed, committed=True)
  File "C:\Users\pc\Spine-Toolbox\spinetoolbox\spine_db_editor\mvcmodels\compound_models.py", line 378, in _add_items
    model.reset_model(ids)
  File "C:\Users\pc\Spine-Toolbox\spinetoolbox\spine_db_editor\mvcmodels\single_models.py", line 28, in reset_model
    self._main_data = sorted(main_data, key=self._sort_key)
TypeError: '<' not supported between instances of 'int' and 'str'
Traceback (most recent call last):
  File "C:\Users\pc\Spine-Toolbox\spinetoolbox\helpers.py", line 136, in new_function
    return func(*args, **kwargs)
  File "C:\Users\pc\Spine-Toolbox\spinetoolbox\fetch_parent.py", line 99, in _apply_pending_changes
    last_handler({db_map: items})
  File "C:\Users\pc\Spine-Toolbox\spinetoolbox\fetch_parent.py", line 322, in handle_items_added
    self._handle_items_added(db_map_data)
  File "C:\Users\pc\Spine-Toolbox\spinetoolbox\spine_db_editor\mvcmodels\compound_models.py", line 343, in handle_items_added
    self._add_items(db_map, entity_class_id, ids_committed, committed=True)
  File "C:\Users\pc\Spine-Toolbox\spinetoolbox\spine_db_editor\mvcmodels\compound_models.py", line 378, in _add_items
    model.reset_model(ids)
  File "C:\Users\pc\Spine-Toolbox\spinetoolbox\spine_db_editor\mvcmodels\single_models.py", line 28, in reset_model
    self._main_data = sorted(main_data, key=self._sort_key)
TypeError: '<' not supported between instances of 'str' and 'int'
soininen commented 1 month ago

The Traceback does look concerning since it comes from the machinery that adds data to the DB editor. Did I get this right:

e-zaline commented 1 month ago

the connections in yellow should not exist, they are not part of the CSV file entity.csv

soininen commented 1 month ago

I cannot reproduce the error with the simple mockup I can build here. I guess it is not possible to share the your Toolbox project, right? Could you send me that Importer's specification at least? Maybe it can give us some hints of what goes wrong here. Are you purging the database when importing?

e-zaline commented 1 month ago

I can share the project (via our shared folder on Sharepoint), no problem. What should I share exactly? The Input_data database?

soininen commented 1 month ago

It would be best if you could share the entire project folder (maybe delete Exporter's and Tool's output folders first to decrease clutter).

soininen commented 1 month ago

Thanks for the data, I can see what goes wrong now!

e-zaline commented 1 month ago
  • Importing success without errors: yes
  • The connections export OK to FlexTool CSV files: yes
  • The database editor does not show the connections: yes and no: it shows the connections but without the associated parameters AND it seems to show connection that do not exist, that were part of previous runs:
image

the connections in yellow should not exist, they are not part of the CSV file entity.csv

For the connections that "should not appear", they do exist in the database, so it's normal that they are in the Editor - my mistake. But as they were not active entity, I guess this is why they don't appear in the CSV.

soininen commented 1 month ago

The issue should be now fixed. Please update your Toolbox.

e-zaline commented 1 month ago

Great, this is working, thanks!