search

spine-tools / Spine-Database-API

Database interface to Spine generic data model
https://www.tools-for-energy-system-modelling.org/
GNU Lesser General Public License v3.0
6 stars 5 forks source link

Conflict resolution #294

Open manuelma opened 8 months ago

manuelma commented 8 months ago

Recently I made a change so if an external source commits to a DB that is open in DB editor, then the entire mapping is reset and the editor is refreshed. Maybe that change predates this one https://github.com/spine-tools/Spine-Toolbox/issues/2353 by @soininen .

I guess I am coming from a bit different perspective, where the user has dirty (uncommitted) changes in their undo stack and then an external commit happens. If the external commit is adding items that conflict with the user uncommitted changes and then the user attempts to commit, it may be chaos.

So to avoid any complications, I made it so whenever an external commit happens (and is notified to toolbox by the source) I really reset the mapping and the user loses everything.

But if the user doesn't have any changes and the external commit just adds new data, then this solution is perfect. So I don't know, maybe we need to work on a more comprehensive solution for conflict resolution, where we really calculate the difference between the newest contents of the DB and the DatabaseMapping, and ask the user to solve?

soininen commented 8 months ago

The need for comprehensive conflict resolution was apparent while I tried to fix spine-tools/Spine-Toolbox#2353. I almost started to write a db_diff module there!

As a first baby step we could give users two options:

A more comprehensive "diff editor" could come later.

We also now have has_external_commits() method in DatabaseMapping which could be useful for checking if conflict resolution is required.

jkiviluo commented 8 months ago

I think the data structure in memory should have additional flags that help us to maintain and recognize conflicting data. This is from my earlier notes (I believe discussed with @manuelma at some point), but entities, entity_classes and parameter values (or maybe even data in all tables) should have a status_flag when in memory. Preliminary options for the flag:

When trying to commit to DB and there is a conflict (because someone has updated the DB meanwhile), data from updated or new could be flagged as stranded, and the user could see what has been stranded while also seeing the conflicting data from the DB - allowing the user to make choices what data to keep (we could also allow the user to dump the stranded data into another DB for convenience and later comparison - this could also be easier first step from the implementation perspective).

For scripts using the API, commit API function could return the set of stranded data (or all commit data) when there is a conflict. Probably the possibility of a conflict has to be resolved first before trying to actually write anything into db. First step is has_external_commits, but if that's true, then checking if the data could be committed without a conflict.