search

spine-tools / Spine-Toolbox

Spine Toolbox is an open source Python package to manage data, scenarios and workflows for modelling and simulation. You can have your local workflow, but work as a team through version control and SQL databases.
https://www.tools-for-energy-system-modelling.org/
GNU Lesser General Public License v3.0
72 stars 17 forks source link

Warning Users about Reusing Julia Consoles #2071

Open DillonJ opened 1 year ago

DillonJ commented 1 year ago

I feel that reuse of Julia consoles is somewhat problematic.

I have had it cause issues that were difficult to diagnose and unexpected.

A problem is that it is on the one hand, useful that a Julia tool runs more quickly the second time, but on the other hand, it can result in old data being used.

I have also seen issues here where restarting the Julia console is the solution to the problem but this isn't at all obvious to a new user.

Perhaps it is particularly an issue with Spine Interface. I have rerun tools with updated data but it looks like some old version of a DB is stored in memory and is trumping the new version.

Perhaps we need to at least alert users to this in some way - perhaps Spine Interface should issue a warning to users when existing in-memory data is somehow trumping the sqlite file which may have been updated.

Sorry I can't be more specific but I thought it would be good to at least raise this here.

manuelma commented 1 year ago

This is probably more related to spinedb_api. Since recently spinedb_api keeps a cache of the DB in the memory and maybe we are failing to invalidate it.

Edit: actually I don't think this is the cause here. SpineInterface reads the full database from 'disk' so it can't be a cache invalidation issue.

jkiviluo commented 1 year ago

I wonder if https://github.com/spine-tools/Spine-Toolbox/issues/2154 has improved the situation?

DillonJ commented 1 year ago

I think this is a Julia specific thing and something to be aware of generally with Julia. What has helped is the toolbox option to automatically kill consoles. Maybe just something that needs to be mentioned in the documentation somewhere?