brightway-lca / brightway2-data

Tools for the management of inventory databases and impact assessment methods. Part of the Brightway LCA framework.
https://docs.brightway.dev/
BSD 3-Clause "New" or "Revised" License
8 stars 23 forks source link

missing vacuum attribute #149

Closed ccomb closed 1 year ago

ccomb commented 1 year ago
Traceback (most recent call last):
  File "/opt/conda/lib/python3.11/site-packages/bw2data/backends/base.py", line 522, in write
    self._efficient_write_many_data(data)
  File "/opt/conda/lib/python3.11/site-packages/bw2data/backends/base.py", line 458, in _efficient_write_many_data
    self.delete(keep_params=True, warn=False)
  File "/opt/conda/lib/python3.11/site-packages/bw2data/backends/base.py", line 611, in delete
    vacuum_needed = len(self) > 500 and vacuum
                                        ^^^^^^
NameError: name 'vacuum' is not defined

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/jovyan/ecobalyse/data/import_agribalyse.py", line 287, in <module>
    main()
  File "/home/jovyan/ecobalyse/data/import_agribalyse.py", line 281, in main
    import_agribalyse()
  File "/home/jovyan/ecobalyse/data/import_agribalyse.py", line 188, in import_agribalyse
    agribalyse.add_unlinked_flows_to_biosphere_database(biosphere)
  File "/opt/conda/lib/python3.11/site-packages/bw2io/importers/base_lci.py", line 420, in add_unlinked_flows_to_biosphere_database
    bio.write(data)
  File "/opt/conda/lib/python3.11/site-packages/bw2data/backends/base.py", line 525, in write
    self.delete(warn=False)
  File "/opt/conda/lib/python3.11/site-packages/bw2data/backends/base.py", line 611, in delete
    vacuum_needed = len(self) > 500 and vacuum
                                        ^^^^^^
NameError: name 'vacuum' is not defined
ccomb commented 1 year ago

After enabling vacuum, I get again this issue:

Traceback (most recent call last):
  File "/home/jovyan/ecobalyse/data/import_ecoinvent.py", line 61, in <module>
    main()
  File "/home/jovyan/ecobalyse/data/import_ecoinvent.py", line 55, in main
    import_ecoinvent()
  File "/home/jovyan/ecobalyse/data/import_ecoinvent.py", line 25, in import_ecoinvent
    ecoinvent.add_unlinked_flows_to_biosphere_database()
  File "/opt/conda/lib/python3.11/site-packages/bw2io/importers/base_lci.py", line 420, in add_unlinked_flows_to_biosphere_database
    bio.write(data)
  File "/opt/conda/lib/python3.11/site-packages/bw2data/backends/base.py", line 522, in write
    self._efficient_write_many_data(data)
  File "/opt/conda/lib/python3.11/site-packages/bw2data/backends/base.py", line 458, in _efficient_write_many_data
    self.delete(keep_params=True, warn=False)
  File "/opt/conda/lib/python3.11/site-packages/bw2data/backends/base.py", line 645, in delete
    sqlite3_lci_db.vacuum()
  File "/opt/conda/lib/python3.11/site-packages/bw2data/sqlite.py", line 49, in vacuum
    self.execute_sql("VACUUM;")
  File "/opt/conda/lib/python3.11/site-packages/bw2data/sqlite.py", line 42, in execute_sql
    return self.db.execute_sql(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.11/site-packages/peewee.py", line 3249, in execute_sql
    with __exception_wrapper__:
  File "/opt/conda/lib/python3.11/site-packages/peewee.py", line 3019, in __exit__
    reraise(new_type, new_type(exc_value, *exc_args), traceback)
  File "/opt/conda/lib/python3.11/site-packages/peewee.py", line 192, in reraise
    raise value.with_traceback(tb)
  File "/opt/conda/lib/python3.11/site-packages/peewee.py", line 3251, in execute_sql
    cursor.execute(sql, params or ())
peewee.OperationalError: cannot VACUUM from within a transaction
cmutel commented 1 year ago

@ccomb What version are you using? I thought this was fixed in https://github.com/brightway-lca/brightway2-data/commit/78b21a611e6961966a073a6ad646769af4f1e42c, but maybe not.

ccomb commented 1 year ago

It was fixed outside delete(), but when executing the delete() function, we might be in a transaction. I pushed a possible fix in this branch with #150

ccomb commented 1 year ago

I'm using the latest, 4.0.DEV23

cmutel commented 1 year ago

Your help is greatly appreciated on DEV23, but for the time being I would switch back to DEV21, both 22 and 23 are unstable and have been yanked from pypi. The refactoring out of bw_projects still needs some polishing.

ccomb commented 1 year ago

:-/ Oops that's bad news, I have already switched to (a minimal fork based on) DEV23 and I am able to run the whole ecobalyse data generation without much problem

mfastudillo commented 1 year ago

... I am experiencing problems with DEV21. AttributeError: 'SQLiteBackend' object has no attribute 'delete_data' .. it seems this method was part of DEV22.. ?

ccomb commented 1 year ago

I've moved to DEV24 and it works for my usage. Closing this issue as it is fixed with #150 @mfastudillo did you try DEV24?

mfastudillo commented 1 year ago

no I did not. For the moment I'll stick to dev18

cmutel commented 1 year ago

Sorry about all the frustration, I merged some code from others that I thought was well tested, but it wasn't integration tested by someone who knows the problem domain. The latest release (DEV24) should be OK, it basically removed the stuff that broke.