search

brightway-lca / brightway2-io

Importing and exporting for the Brightway LCA framework
BSD 3-Clause "New" or "Revised" License
26 stars 40 forks source link

Some extractor methods do not have the `use_mp` flag implemented, causing them to fail. #233

Open riftia opened 7 months ago

riftia commented 7 months ago

When trying to import a database with bw2io.SingleOutputEcospold1Importer, there is a flag called use_mp that controls whether multiprocessing will be used or not. This flag is inherited by the default Ecospold1LCIAExtractor extractor class. But if you want to change the default extractor to something else, for example, Ecospold1LCIAExtractor or the JSONLDExtractor, the import process fails because use_mp flag is not implemented in these non-default extractors.

>>> import bw2io
>>> from bw2io.extractors import ecospold1_lcia

>>> db_name = "eco38"
>>> importer = bw2io.SingleOutputEcospold1Importer(filepath=f, db_name=db_name, extractor=ecospold1_lcia.Ecospold1LCIAExtractor)

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
[/home/riftia/repo/p3op/test2.ipynb](https://file+.vscode-resource.vscode-cdn.net/home/riftia/repo/p3op/test2.ipynb) Cell 11 line 1
----> [1](vscode-notebook-cell:/home/riftia/repo/p3op/test2.ipynb#X21sZmlsZQ%3D%3D?line=0) i = bw2io.SingleOutputEcospold1Importer(filepath=f, db_name="eco38", extractor=ecospold1_lcia.Ecospold1LCIAExtractor)
      [2](vscode-notebook-cell:/home/riftia/repo/p3op/test2.ipynb#X21sZmlsZQ%3D%3D?line=1) i.statistics()

File [~/repo/p3op/.venv/lib/python3.10/site-packages/bw2io/importers/ecospold1.py:85](https://file+.vscode-resource.vscode-cdn.net/home/riftia/repo/p3op/~/repo/p3op/.venv/lib/python3.10/site-packages/bw2io/importers/ecospold1.py:85), in SingleOutputEcospold1Importer.__init__(self, filepath, db_name, use_mp, extractor)
     [83](https://file+.vscode-resource.vscode-cdn.net/home/riftia/repo/p3op/~/repo/p3op/.venv/lib/python3.10/site-packages/bw2io/importers/ecospold1.py:83) start = time()
     [84](https://file+.vscode-resource.vscode-cdn.net/home/riftia/repo/p3op/~/repo/p3op/.venv/lib/python3.10/site-packages/bw2io/importers/ecospold1.py:84) try:
---> [85](https://file+.vscode-resource.vscode-cdn.net/home/riftia/repo/p3op/~/repo/p3op/.venv/lib/python3.10/site-packages/bw2io/importers/ecospold1.py:85)     self.data = extractor.extract(filepath, db_name, use_mp=use_mp)
     [86](https://file+.vscode-resource.vscode-cdn.net/home/riftia/repo/p3op/~/repo/p3op/.venv/lib/python3.10/site-packages/bw2io/importers/ecospold1.py:86) except RuntimeError as e:
     [87](https://file+.vscode-resource.vscode-cdn.net/home/riftia/repo/p3op/~/repo/p3op/.venv/lib/python3.10/site-packages/bw2io/importers/ecospold1.py:87)     raise MultiprocessingError(
     [88](https://file+.vscode-resource.vscode-cdn.net/home/riftia/repo/p3op/~/repo/p3op/.venv/lib/python3.10/site-packages/bw2io/importers/ecospold1.py:88)         "Multiprocessing error; re-run using `use_mp=False`"
     [89](https://file+.vscode-resource.vscode-cdn.net/home/riftia/repo/p3op/~/repo/p3op/.venv/lib/python3.10/site-packages/bw2io/importers/ecospold1.py:89)     ).with_traceback(e.__traceback__)

TypeError: Ecospold1LCIAExtractor.extract() got an unexpected keyword argument 'use_mp'
cmutel commented 7 months ago

This should be fixed in 28b9d768fa0e6212ca6c96ba550b88e46a0b5cc6; @riftia could you check to make sure it is aligned with your expectations?