Update dependencies

[rouille]

Pull Request doc

Purpose

Update dependencies as new packages were introduced with the building electrification project. It was tricky. Also, docstrings have been cleaned up and missing __init__.py files have been added.

What the code is doing

N/A

Testing

Current unit test

Where to look

• Impossible to generate a Pipefile.lock file that work with all 3 Python versions we support. The Pipefile.lock file in this PR has been generated with Python 3.10.4. For this reason, dependencies for testing are installed using the requirements.txt file (see tox.ini).@jenhagg, do we want to keep the Pipefile.lock in version control? pipenv recommends not to if multiple version of python are targeted.
• Skip test in prereise.gather.flexibilitydata.doe.tests.test_bus_data that most of the time fails to connect to DOE server
• Add missing __init__.py files

Usage Example/Visuals

N/A

Time estimate

15min

[jenhagg]

I would keep Pipfile.lock versioned. Maybe also add the python version to Pipfile (see docs), as below. Btw, best source I found for python_full_version is in PEP 508.

[requires]
python_full_version= "3.10.4"

I think testing multiple python versions with requirements.txt is fine. Not 100% reproducible, but don't expect too much noise.

Thinking out loud a bit here, but maybe helpful:

The context for Pipfile.lock is that pipenv and lock file is designed for "deployment environments" meaning an application that will always use exactly one python version. We kinda have a hybrid situation where we publish packages which are libraries - meant to be usable in a variable number of python environments (could be multiple OSes, python versions, etc. whatever we specify is supported). And pipenv says it's not [necessarily] meant for packages. But also have users who work with the packages directly in a local environment, but not e.g. typically included in another application, the way libraries "normally" are (idk, just in my experience). I think this is somewhat a result of splitting up repositories and simplifying internal dev workflow, e.g. by setting powersimdata as a dependency in postreise. But there is also value in making the functionality accessible to anyone generally.

So anyway, since we more or less want both a reproducible environment and to support package distribution, it maybe makes sense to keep both Pipfile.lock and requirements.txt, respectively. And the CI checks will catch anything that breaks downstream users (fine print: as long as it's covered by tests/linting/etc).

[rouille]

I would keep Pipfile.lock versioned. Maybe also add the python version to Pipfile (see docs), as below. Btw, best source I found for python_full_version is in PEP 508.

[requires]
python_full_version= "3.10.4"

I think testing multiple python versions with requirements.txt is fine. Not 100% reproducible, but don't expect too much noise.

Thinking out loud a bit here, but maybe helpful:

The context for Pipfile.lock is that pipenv and lock file is designed for "deployment environments" meaning an application that will always use exactly one python version. We kinda have a hybrid situation where we publish packages which are libraries - meant to be usable in a variable number of python environments (could be multiple OSes, python versions, etc. whatever we specify is supported). And pipenv says it's not [necessarily] meant for packages. But also have users who work with the packages directly in a local environment, but not e.g. typically included in another application, the way libraries "normally" are (idk, just in my experience). I think this is somewhat a result of splitting up repositories and simplifying internal dev workflow, e.g. by setting powersimdata as a dependency in postreise. But there is also value in making the functionality accessible to anyone generally.

So anyway, since we more or less want both a reproducible environment and to support package distribution, it maybe makes sense to keep both Pipfile.lock and requirements.txt, respectively. And the CI checks will catch anything that breaks downstream users (fine print: as long as it's covered by tests/linting/etc).

Sounds good. I have updated the Pipfile as suggested.