spine-tools / Spine-Database-API

Database interface to Spine generic data model
https://www.tools-for-energy-system-modelling.org/
GNU Lesser General Public License v3.0
6 stars 5 forks source link

Committing to database from code fails every 2nd try #440

Closed jkiviluo closed 3 months ago

jkiviluo commented 3 months ago

I'm trying to purge an existing database with data, write mostly the same data there again, and then commit. However, there is something funny going on, since it works every 2nd time that I run the code. Every other time it fails. I finally caught a traceback that might be useful in tracking the cause of this down:

Traceback (most recent call last):
  File "C:\Users\prokjt\AppData\Roaming\Python\Python39\site-packages\sqlalchemy\engine\base.py", line 1256, in _execute_context
    self.dialect.do_executemany(
  File "C:\Users\prokjt\AppData\Roaming\Python\Python39\site-packages\sqlalchemy\engine\default.py", line 605, in do_executemany
    cursor.executemany(statement, parameters)
sqlite3.IntegrityError: NOT NULL constraint failed: entity_alternative.alternative_id

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "C:\data\spi-entity\Spine-Toolbox\src\spinedb-api\spinedb_api\db_mapping_commit_mixin.py", line 59, in _do_add_items
    connection.execute(table.insert(), [x.resolve() for x in temp_id_items])
  File "C:\Users\prokjt\AppData\Roaming\Python\Python39\site-packages\sqlalchemy\engine\base.py", line 1011, in execute
    return meth(self, multiparams, params)
  File "C:\Users\prokjt\AppData\Roaming\Python\Python39\site-packages\sqlalchemy\sql\elements.py", line 298, in _execute_on_connection
    return connection._execute_clauseelement(self, multiparams, params)
  File "C:\Users\prokjt\AppData\Roaming\Python\Python39\site-packages\sqlalchemy\engine\base.py", line 1124, in _execute_clauseelement
    ret = self._execute_context(
  File "C:\Users\prokjt\AppData\Roaming\Python\Python39\site-packages\sqlalchemy\engine\base.py", line 1316, in _execute_context
    self._handle_dbapi_exception(
  File "C:\Users\prokjt\AppData\Roaming\Python\Python39\site-packages\sqlalchemy\engine\base.py", line 1510, in _handle_dbapi_exception
    util.raise_(
  File "C:\Users\prokjt\AppData\Roaming\Python\Python39\site-packages\sqlalchemy\util\compat.py", line 182, in raise_
    raise exception
  File "C:\Users\prokjt\AppData\Roaming\Python\Python39\site-packages\sqlalchemy\engine\base.py", line 1256, in _execute_context
    self.dialect.do_executemany(
  File "C:\Users\prokjt\AppData\Roaming\Python\Python39\site-packages\sqlalchemy\engine\default.py", line 605, in do_executemany
    cursor.executemany(statement, parameters)
sqlalchemy.exc.IntegrityError: (sqlite3.IntegrityError) NOT NULL constraint failed: entity_alternative.alternative_id
[SQL: INSERT INTO entity_alternative (id, entity_id, alternative_id, active, commit_id) VALUES (?, ?, ?, ?, ?)]
[parameters: ((1, 1, None, 1, 1576), (2, None, None, 1, 1576), (3, None, None, 1, 1576))]
(Background on this error at: http://sqlalche.me/e/13/gkpj)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "C:\data\Spine\ines-osemosys\ines-osemosys\osemosys_to_ines.py", line 447, in <module>
    main()
  File "C:\data\Spine\ines-osemosys\ines-osemosys\osemosys_to_ines.py", line 62, in main
    target_db = create_periods(source_db, target_db)
  File "C:\data\Spine\ines-osemosys\ines-osemosys\osemosys_to_ines.py", line 227, in create_periods
    target_db.commit_session("Added periods from YEARs to ines_db")
  File "C:\data\spi-entity\Spine-Toolbox\src\spinedb-api\spinedb_api\db_mapping.py", line 821, in commit_session
    self._do_add_items(connection, tablename, *to_add)
  File "C:\data\spi-entity\Spine-Toolbox\src\spinedb-api\spinedb_api\db_mapping_commit_mixin.py", line 67, in _do_add_items
    raise SpineDBAPIError(msg) from e
spinedb_api.exception.SpineDBAPIError: DBAPIError while inserting entity_alternative items: ('NOT NULL constraint failed: entity_alternative.alternative_id',)

Process finished with exit code 1
soininen commented 3 months ago

This was fixed by commits 2f1b1c4576a32ed53b1ebd0a4d6c76b68e74d633, 25a596e55054dabe50d638fe6a1a0b98d548625b and aa332c434b6cb021ea433b31d22cb21c12045f4e