pip install errors

[michael-long88]

While running the command pip install git+https://github.com/USEPA/standardizedinventories@v0.9.8#egg=standardizedinventories (also tried 0.9.7), I'm receiving the following error:

WARNING: Generating metadata for package standardizedinventories produced metadata for project name stewi. Fix your #egg=standardizedinventories fragments.
WARNING: Discarding git+https://github.com/USEPA/standardizedinventories@v0.9.7#egg=standardizedinventories. Requested stewi from git+https://github.com/USEPA/standardizedinventories@v0.9.7#egg=standardizedinventories has inconsistent name: filename has 'standardizedinventories', but metadata has 'StEWI'
ERROR: Could not find a version that satisfies the requirement standardizedinventories (unavailable) (from versions: none)
ERROR: No matching distribution found for standardizedinventories (unavailable)

Using python 3.8.10

[bl-young]

Sorry the wiki is not correct. Please try pip install git+https://github.com/USEPA/standardizedinventories@v0.9.8#egg=StEWI

or for the current head pip install git+https://github.com/USEPA/standardizedinventories#egg=StEWI

Note this package is still under development but we are preparing for a version 1.0.0 release soon.

[michael-long88]

I'm not getting any errors on installation now, but when I try to test the installation by importing one of the libraries, I get the same error no matter which one I try to import

>>> import stewi
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/michaellong/miniconda/envs/epa/lib/python3.8/site-packages/stewi/__init__.py", line 11, in <module>
    from stewi.globals import get_required_fields, filter_inventory,\
  File "/Users/michaellong/miniconda/envs/epa/lib/python3.8/site-packages/stewi/globals.py", line 18, in <module>
    from esupy.processed_data_mgmt import Paths, FileMeta,\
ImportError: cannot import name 'read_source_metadata' from 'esupy.processed_data_mgmt' (/Users/michaellong/miniconda/envs/epa/lib/python3.8/site-packages/esupy/processed_data_mgmt.py)

[bl-young]

It appears as though the version of esupy (0.1.2) required by stewi in the v0.9.8 release was one behind. Upgrade esupy using pip install git+https://github.com/USEPA/esupy#egg=esupy -U (I think this syntax is correct)

or reinstall StEWI from the current head (not from version 0.9.8) pip install git+https://github.com/USEPA/standardizedinventories#egg=StEWI

[michael-long88]

Looks like the upgrade worked. In the release notes, it says "Processed stewi files are no longer maintained in the repository and must be generated locally". Is there documentation on generating those locally? Trying to follow along in examples/stewiandstewicomboexample.py, but when I run stewi.seeAvailableInventoriesandYears(), I'm getting

ERROR directory not found: /Users/michaellong/Library/Application Support/stewi/flowbyfacility/
flowbyfacility inventories available (name, year):
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/michaellong/miniconda/envs/epa/lib/python3.8/site-packages/stewi/__init__.py", line 62, in seeAvailableInventoriesandYears
    for i in existing_inventories.keys():
AttributeError: 'NoneType' object has no attribute 'keys'

[bl-young]

Yes the documentation still needs to be updated with the changes we have made. For generating from the command line, you can try something like:

python -m stewi.GHGRP A -y 2017

Each module describes the options that can be passed, e.g. https://github.com/USEPA/standardizedinventories/blob/master/stewi/GHGRP.py#L5-L29

[michael-long88]

It looks like https://github.com/USEPA/standardizedinventories/blob/master/stewi/GHGRP.py#L54 is still referencing the data folder as GHGRP/, but the folder name is actually ghgrp/ which is causing a FileNotFoundError

[bl-young]

See this fix for the folder issue https://github.com/USEPA/standardizedinventories/pull/77

[michael-long88]

Ok so I checked my local stewi/data/ and I noticed it has NEI/, DMR/, TRI/, and eGRID/, but not RCRAInfo/ or GHGRP/, which would explain why I'm still getting the FileNotFoundError after running pip install again from head. I'm not sure if I missed a step or not as to why it's missing those 2 folders

[bl-young]

@michael-long88 This will fix it: https://github.com/USEPA/standardizedinventories/commit/91dfdb90586213a85d4dfefe778c790a1b1a6c5b

[michael-long88]

It now seems to be breaking while downloading the data. It might even be breaking when getting the table URL since I'm not seeing the log statement that proceeds the generate_url function call. Since it's breaking somewhere in there, it's causing the try block to fail, which means that table_df isn't actually assigned, resulting in the following error:

➜ python -m stewi.GHGRP A -Y 2019
INFO downloading and processing GHGRP data to /Users/michaellong/Library/Application Support/stewi/GHGRP Data Files/tables/2019/
INFO Downloading C_CONFIGURATION_LEVEL_INFO (rows: 15873)
Traceback (most recent call last):
  File "/Users/michaellong/miniconda/envs/epa/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/Users/michaellong/miniconda/envs/epa/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/Users/michaellong/miniconda/envs/epa/lib/python3.8/site-packages/stewi/GHGRP.py", line 813, in <module>
    main()
  File "/Users/michaellong/miniconda/envs/epa/lib/python3.8/site-packages/stewi/GHGRP.py", line 665, in main
    ghgrp1 = download_and_parse_subpart_tables(year)
  File "/Users/michaellong/miniconda/envs/epa/lib/python3.8/site-packages/stewi/GHGRP.py", line 304, in download_and_parse_subpart_tables
    table_df = import_or_download_table(filepath, subpart_emissions_table,
  File "/Users/michaellong/miniconda/envs/epa/lib/python3.8/site-packages/stewi/GHGRP.py", line 259, in import_or_download_table
    for col in table_df:
UnboundLocalError: local variable 'table_df' referenced before assignment

Some error statements in those blocks of code would also be useful for tracking down errors for those of us running from a pip install rather than running directly from a local copy of the repo.

[WesIngwersen]

@michael-long88 please create new issues rather than adding new comments to this same issue as you continue to explore and test StEWI. I'm creating a new issue from your latest comment so we can continue the discussion there.