search

spine-tools / Spine-Toolbox

Spine Toolbox is an open source Python package to manage data, scenarios and workflows for modelling and simulation. You can have your local workflow, but work as a team through version control and SQL databases.
https://www.tools-for-energy-system-modelling.org/
GNU Lesser General Public License v3.0
69 stars 17 forks source link

Auto-Commit Option #2241

Open DillonJ opened 1 year ago

DillonJ commented 1 year ago

I have experienced the issue where data fails to commit in DB Editor quite a lot. It's something that seems to get fixed and then reoccurs. It's very frustrating to lose data. Below is an example:

https://github.com/spine-tools/Spine-Toolbox/issues/2216

Over time you develop the habit to commit more frequently, actually after each individual change. So committing in this case actually becomes redundant.

Could we have an auto-commit option? I would definitely use it if we had it, to defend against this issue.

Thoughts @jkiviluo @manuelma?

jkiviluo commented 1 year ago

I think we should fix the issues that cause Toolbox to crash. One should be able to do a set of changes that relate to each other and only then commit with an informative message about all of those changes. Auto-committing would make people skimp on the documentation of their data, which can bite them later.

Or do you mean that when trying to run an importer to modify the data when the database has uncommitted changes, would fire up the commit widget? That would be good.

DillonJ commented 1 year ago

Rather than crashing I was thinking more of Toolbox issues which cause a commit to fail. Some causes of these I have seen in the past are a locked DB and SQL warnings from SQLAlchemy usually related to a what-you-are-seeing-is-not-what's-really-there cache issues. The main thing is that with autocommit, a user would see when there's a problem immediately and this would reduce the amount of work lost.

I guess people wouldn't have to use it. There are different use cases I guess... a collaborative DB with multiple users working on it where it wouldn;t be appropriate - or individuals trying to build up a data set where committing is an inconvenience and a risk

You could have an override on a DB that greys out the option perhaps when auto-commit is to be disallowed.

jkiviluo commented 1 year ago

Ok, I understand better why you might want auto-commit. I still think we should aim to catch the different situations in proper way. If there are two sources of edits, the latter one should not be allowed to proceed before the first one has finished. We have tried to think about this in the upcoming new memory data structure. It should also enable graceful failing when there is a conflict (wouldn't drop your changes, but would rather highlight the conflicts).

Of course, all of that will take time and it sounds like the auto-commit should be there at least meanwhile. If so, I'd prefer to keep it as an advanced option and maybe with a warning that it may be removed at some point (once we can manage the problems in the right way).

jkiviluo commented 3 months ago

In my experience DB has gotten much more stable in recent times and also conflict resolution has been improved. Is there still need for this or could we close the issue (@DillonJ)?