ccomb
commented
1 year ago Trying with Brightway 2.5, I've managed to progress up to the end of the import once, and the next time I got the VACUUM error again:
Applying strategy: link_iterable_by_fields
18641 datasets
5064938 exchanges
0 unlinked exchanges
Not able to determine geocollections for all datasets. This database is not ready for regionalization.
Vacuuming database
Vacuuming database
Traceback (most recent call last):
File "/home/.../data/venv3.11/lib/python3.11/site-packages/peewee.py", line 3236, in execute_sql
cursor.execute(sql, params or ())
sqlite3.OperationalError: cannot VACUUM from within a transaction
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/.../data/food/import_agb/./import_agribalyse.py", line 240, in <module>
agb_importer.write_database()
File "/home/.../data/venv3.11/lib/python3.11/site-packages/bw2io/importers/simapro_csv.py", line 108, in write_database
db = super(SimaProCSVImporter, self).write_database(data, name, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/.../data/venv3.11/lib/python3.11/site-packages/bw2io/importers/base_lci.py", line 276, in write_database
db.write(existing)
File "/home/.../data/venv3.11/lib/python3.11/site-packages/bw2data/project.py", line 446, in writable_project
return wrapped(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/.../data/venv3.11/lib/python3.11/site-packages/bw2data/backends/base.py", line 535, in write
self._efficient_write_many_data(data)
File "/home/.../data/venv3.11/lib/python3.11/site-packages/bw2data/backends/base.py", line 461, in _efficient_write_many_data
self.delete(keep_params=True, warn=False)
File "/home/.../data/venv3.11/lib/python3.11/site-packages/bw2data/project.py", line 446, in writable_project
return wrapped(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/.../data/venv3.11/lib/python3.11/site-packages/bw2data/backends/base.py", line 661, in delete
sqlite3_lci_db.vacuum()
File "/home/.../data/venv3.11/lib/python3.11/site-packages/bw2data/sqlite.py", line 48, in vacuum
self.execute_sql("VACUUM;")
File "/home/.../data/venv3.11/lib/python3.11/site-packages/bw2data/sqlite.py", line 41, in execute_sql
return self.db.execute_sql(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/.../data/venv3.11/lib/python3.11/site-packages/peewee.py", line 3234, in execute_sql
with __exception_wrapper__:
File "/home/.../data/venv3.11/lib/python3.11/site-packages/peewee.py", line 3010, in __exit__
reraise(new_type, new_type(exc_value, *exc_args), traceback)
File "/home/.../data/venv3.11/lib/python3.11/site-packages/peewee.py", line 192, in reraise
raise value.with_traceback(tb)
File "/home/.../data/venv3.11/lib/python3.11/site-packages/peewee.py", line 3236, in execute_sql
cursor.execute(sql, params or ())
peewee.OperationalError: cannot VACUUM from within a transaction
The set of versions I'm using is: brightway25-1.0.6 bw2analyzer-0.11.4 bw2calc-2.0.dev12 bw2data-4.0.dev19 bw2io-0.9.dev10 bw2parameters-1.0.0 bw_migrations-0.2 bw_processing-0.8.2 through this requirements.txt:
git+https://github.com/brightway-lca/brightway25
git+https://github.com/brightway-lca/brightway2-calc
git+https://github.com/brightway-lca/brightway2-data@4.0.DEV19
git+https://github.com/ccomb/brightway2-io
git+https://github.com/brightway-lca/brightway2-parameters
fjson==0.1.6
matplotlib==3.6.3
numpy==1.24.2
olca-ipc==0.0.12
pandas==1.5.3
scipy==1.10.1
xlrd==2.0.1
romainsacchi
Something new at the end of the import of agribalyse 3.1.1:
Looks like there is something wrong trying to delete data with VACUUM inside a transaction. I'm on the
bw2legacybranch: https://github.com/brightway-lca/brightway2-data/blob/bw2legacy/bw2data/backends/peewee/database.py#L261There is a first Exception in
self._efficient_write_many_data(data)which does not appear and is then shadowed by a second exception due to vacuumin inside the transaction inself.delete(warn=False).If I try to remove the try: except: to trigger the first exception, I fall on a second one elsewhere similar to the first one:
If I remove the
self.deleteto understand I finally fall on the original Exception, which is a constraint error: