search

ProjectDrawdown / solutions

The mission of Project Drawdown is to help the world reach “Drawdown”— the point in the future when levels of greenhouse gases in the atmosphere stop climbing and start to steadily decline, thereby stopping catastrophic climate change — as quickly, safely, and equitably as possible.
https://www.drawdown.org/
Other
211 stars 91 forks source link

Units #525

Closed gerald-scharitzer closed 2 years ago

gerald-scharitzer commented 2 years ago

Decorate Functions with their Units

This shall close issue 361.

Add the library units-of-measure for SI units and extend it with frequently used non-SI units in tools/units.py.

Update your conda environment with conda env update -n pd-dev.

Situation

Solution modules specify their units in the dictionary named units as in solution/airplanes/__init__.py.

Several model functions utilize a naming convention, where the second qualifier (first_second_*) specifies the unit as in co2eq_mmt_reduced, where mmt specifies million metric tonnes.

Rationale

Decorate functions or data frames with their units, because this can be done independent of naming conventions, and types of parameters and return values. This will help us to keep stable interfaces once we reach version 1.0.0.

Function names and docstrings are unstructured and would have to be parsed based on some to-be-defined grammar.

Attributes on functions and data frames can be more structured and machine-readable to also serve UIs for labeling table columns and chart axes.

Pandas does not provide direct support for units. There is an open issue for this since 2015 and to me it does not look like it is going to be resolved anytime soon.

Since not only data frames can have units (more specifically the data series therein), but also functions, attributes, and variables, I do not want to inject a units attribute into all of those or subclass them.

Enhancement

Map objects (including functions) to units as demonstrated in units.ipynb.

Units have identities and attributes with user-defined types.

Pytest Before

========================================================================================= short test summary info =========================================================================================
FAILED solution/test_factory.py::test_load_custom_scenario_by_copying - zipfile.BadZipFile: File is not a zip file
FAILED solution/afforestation/tests/test_afforestation.py::test_key_results - AssertionError: 
FAILED solution/bamboo/tests/test_bamboo.py::test_key_results - AssertionError: 
FAILED solution/biogas/tests/test_biogas.py::test_deep_results - AssertionError: 
FAILED solution/biogas_small/tests/test_biogas_small.py::test_deep_results - AssertionError: 
FAILED solution/bioplastic/tests/test_bioplastic.py::test_key_results - AssertionError: 
FAILED solution/bioplastic/tests/test_bioplastic.py::test_deep_results - AssertionError: 
FAILED solution/conservationagriculture/tests/test_conservationagriculture.py::test_deep_results - AssertionError: 
FAILED solution/hcrecycling/tests/test_hcrecycling.py::test_deep_results - AssertionError: 
FAILED solution/hybridcars/tests/test_hybridcars.py::test_loader - zipfile.BadZipFile: File is not a zip file
FAILED solution/hybridcars/tests/test_hybridcars.py::test_key_results - zipfile.BadZipFile: File is not a zip file
FAILED solution/hybridcars/tests/test_hybridcars.py::test_deep_results - zipfile.BadZipFile: File is not a zip file
FAILED solution/improvedcookstoves/tests/test_improvedcookstoves.py::test_deep_results - AssertionError: 
FAILED solution/improvedrice/tests/test_improvedrice.py::test_deep_results - AssertionError: 
FAILED solution/landfillmethane/tests/test_landfillmethane.py::test_deep_results - AssertionError: 
FAILED solution/methaneleak/tests/test_methaneleak.py::test_deep_results - AssertionError: 
FAILED solution/peatlands/tests/test_peatlands.py::test_deep_results - AssertionError: 
FAILED solution/recycledpaper/tests/test_recycledpaper.py::test_deep_results - AssertionError: 
FAILED solution/riceintensification/tests/test_riceintensification.py::test_deep_results - AssertionError: 
FAILED solution/ships/tests/test_ships.py::test_key_results - AssertionError: 
FAILED solution/ships/tests/test_ships.py::test_deep_results - AssertionError: 
FAILED solution/solarhotwater/tests/test_solarhotwater.py::test_key_results - AssertionError: 
FAILED solution/solarhotwater/tests/test_solarhotwater.py::test_deep_results - AssertionError: 
FAILED solution/trains/tests/test_trains.py::test_loader - zipfile.BadZipFile: File is not a zip file
FAILED solution/trains/tests/test_trains.py::test_key_results - zipfile.BadZipFile: File is not a zip file
FAILED solution/trains/tests/test_trains.py::test_deep_results - zipfile.BadZipFile: File is not a zip file
FAILED solution/wastetoenergy/tests/test_wastetoenergy.py::test_deep_results - AssertionError: 
FAILED solution/waterefficiency/tests/test_waterefficiency.py::test_deep_results - AssertionError: 
================================================================== 28 failed, 1322 passed, 12 skipped, 84 warnings in 764.12s (0:12:44) ===================================================================

Update

git checkout -q units
conda env update -n pd-dev

Pytest After

========================================================================================= short test summary info =========================================================================================
FAILED solution/test_factory.py::test_load_custom_scenario_by_copying - zipfile.BadZipFile: File is not a zip file
FAILED solution/afforestation/tests/test_afforestation.py::test_key_results - AssertionError: 
FAILED solution/bamboo/tests/test_bamboo.py::test_key_results - AssertionError: 
FAILED solution/biogas/tests/test_biogas.py::test_deep_results - AssertionError: 
FAILED solution/biogas_small/tests/test_biogas_small.py::test_deep_results - AssertionError: 
FAILED solution/bioplastic/tests/test_bioplastic.py::test_key_results - AssertionError: 
FAILED solution/bioplastic/tests/test_bioplastic.py::test_deep_results - AssertionError: 
FAILED solution/conservationagriculture/tests/test_conservationagriculture.py::test_deep_results - AssertionError: 
FAILED solution/hcrecycling/tests/test_hcrecycling.py::test_deep_results - AssertionError: 
FAILED solution/hybridcars/tests/test_hybridcars.py::test_loader - zipfile.BadZipFile: File is not a zip file
FAILED solution/hybridcars/tests/test_hybridcars.py::test_key_results - zipfile.BadZipFile: File is not a zip file
FAILED solution/hybridcars/tests/test_hybridcars.py::test_deep_results - zipfile.BadZipFile: File is not a zip file
FAILED solution/improvedcookstoves/tests/test_improvedcookstoves.py::test_deep_results - AssertionError: 
FAILED solution/improvedrice/tests/test_improvedrice.py::test_deep_results - AssertionError: 
FAILED solution/landfillmethane/tests/test_landfillmethane.py::test_deep_results - AssertionError: 
FAILED solution/methaneleak/tests/test_methaneleak.py::test_deep_results - AssertionError: 
FAILED solution/peatlands/tests/test_peatlands.py::test_deep_results - AssertionError: 
FAILED solution/recycledpaper/tests/test_recycledpaper.py::test_deep_results - AssertionError: 
FAILED solution/riceintensification/tests/test_riceintensification.py::test_deep_results - AssertionError: 
FAILED solution/ships/tests/test_ships.py::test_key_results - AssertionError: 
FAILED solution/ships/tests/test_ships.py::test_deep_results - AssertionError: 
FAILED solution/solarhotwater/tests/test_solarhotwater.py::test_key_results - AssertionError: 
FAILED solution/solarhotwater/tests/test_solarhotwater.py::test_deep_results - AssertionError: 
FAILED solution/trains/tests/test_trains.py::test_loader - zipfile.BadZipFile: File is not a zip file
FAILED solution/trains/tests/test_trains.py::test_key_results - zipfile.BadZipFile: File is not a zip file
FAILED solution/trains/tests/test_trains.py::test_deep_results - zipfile.BadZipFile: File is not a zip file
FAILED solution/wastetoenergy/tests/test_wastetoenergy.py::test_deep_results - AssertionError: 
FAILED solution/waterefficiency/tests/test_waterefficiency.py::test_deep_results - AssertionError: 
================================================================== 28 failed, 1322 passed, 12 skipped, 84 warnings in 782.05s (0:13:02) ===================================================================
denised commented 2 years ago

I do have one request: please move the units.py file to the model directory. Originally code was split between the directories in a more arbitrary way, but at this point we consistently have code that supports the solution "runtime" in the model directory, and code that supports creation and testing of models in the tools directory. ...thanks!

gerald-scharitzer commented 2 years ago

Done, merged changes from develop, and re-tested

Pytest Before

========================================================================================= short test summary info =========================================================================================
FAILED solution/test_factory.py::test_load_custom_scenario_by_copying - zipfile.BadZipFile: File is not a zip file
FAILED solution/afforestation/tests/test_afforestation.py::test_key_results - AssertionError: 
FAILED solution/bamboo/tests/test_bamboo.py::test_key_results - AssertionError: 
FAILED solution/biogas/tests/test_biogas.py::test_deep_results - AssertionError: 
FAILED solution/biogas_small/tests/test_biogas_small.py::test_deep_results - AssertionError: 
FAILED solution/bioplastic/tests/test_bioplastic.py::test_key_results - AssertionError: 
FAILED solution/bioplastic/tests/test_bioplastic.py::test_deep_results - AssertionError: 
FAILED solution/conservationagriculture/tests/test_conservationagriculture.py::test_deep_results - AssertionError: 
FAILED solution/hcrecycling/tests/test_hcrecycling.py::test_deep_results - AssertionError: 
FAILED solution/hybridcars/tests/test_hybridcars.py::test_loader - zipfile.BadZipFile: File is not a zip file
FAILED solution/hybridcars/tests/test_hybridcars.py::test_key_results - zipfile.BadZipFile: File is not a zip file
FAILED solution/hybridcars/tests/test_hybridcars.py::test_deep_results - zipfile.BadZipFile: File is not a zip file
FAILED solution/improvedcookstoves/tests/test_improvedcookstoves.py::test_deep_results - AssertionError: 
FAILED solution/improvedrice/tests/test_improvedrice.py::test_deep_results - AssertionError: 
FAILED solution/landfillmethane/tests/test_landfillmethane.py::test_deep_results - AssertionError: 
FAILED solution/methaneleak/tests/test_methaneleak.py::test_deep_results - AssertionError: 
FAILED solution/peatlands/tests/test_peatlands.py::test_deep_results - AssertionError: 
FAILED solution/recycledpaper/tests/test_recycledpaper.py::test_deep_results - AssertionError: 
FAILED solution/riceintensification/tests/test_riceintensification.py::test_deep_results - AssertionError: 
FAILED solution/ships/tests/test_ships.py::test_key_results - AssertionError: 
FAILED solution/ships/tests/test_ships.py::test_deep_results - AssertionError: 
FAILED solution/solarhotwater/tests/test_solarhotwater.py::test_key_results - AssertionError: 
FAILED solution/solarhotwater/tests/test_solarhotwater.py::test_deep_results - AssertionError: 
FAILED solution/trains/tests/test_trains.py::test_loader - zipfile.BadZipFile: File is not a zip file
FAILED solution/trains/tests/test_trains.py::test_key_results - zipfile.BadZipFile: File is not a zip file
FAILED solution/trains/tests/test_trains.py::test_deep_results - zipfile.BadZipFile: File is not a zip file
FAILED solution/wastetoenergy/tests/test_wastetoenergy.py::test_deep_results - AssertionError: 
FAILED solution/waterefficiency/tests/test_waterefficiency.py::test_deep_results - AssertionError: 
================================================================== 28 failed, 1322 passed, 12 skipped, 84 warnings in 763.14s (0:12:43) ===================================================================

Pytest After

========================================================================================= short test summary info =========================================================================================
FAILED solution/test_factory.py::test_load_custom_scenario_by_copying - zipfile.BadZipFile: File is not a zip file
FAILED solution/afforestation/tests/test_afforestation.py::test_key_results - AssertionError: 
FAILED solution/bamboo/tests/test_bamboo.py::test_key_results - AssertionError: 
FAILED solution/biogas/tests/test_biogas.py::test_deep_results - AssertionError: 
FAILED solution/biogas_small/tests/test_biogas_small.py::test_deep_results - AssertionError: 
FAILED solution/bioplastic/tests/test_bioplastic.py::test_key_results - AssertionError: 
FAILED solution/bioplastic/tests/test_bioplastic.py::test_deep_results - AssertionError: 
FAILED solution/conservationagriculture/tests/test_conservationagriculture.py::test_deep_results - AssertionError: 
FAILED solution/hcrecycling/tests/test_hcrecycling.py::test_deep_results - AssertionError: 
FAILED solution/hybridcars/tests/test_hybridcars.py::test_loader - zipfile.BadZipFile: File is not a zip file
FAILED solution/hybridcars/tests/test_hybridcars.py::test_key_results - zipfile.BadZipFile: File is not a zip file
FAILED solution/hybridcars/tests/test_hybridcars.py::test_deep_results - zipfile.BadZipFile: File is not a zip file
FAILED solution/improvedcookstoves/tests/test_improvedcookstoves.py::test_deep_results - AssertionError: 
FAILED solution/improvedrice/tests/test_improvedrice.py::test_deep_results - AssertionError: 
FAILED solution/landfillmethane/tests/test_landfillmethane.py::test_deep_results - AssertionError: 
FAILED solution/methaneleak/tests/test_methaneleak.py::test_deep_results - AssertionError: 
FAILED solution/peatlands/tests/test_peatlands.py::test_deep_results - AssertionError: 
FAILED solution/recycledpaper/tests/test_recycledpaper.py::test_deep_results - AssertionError: 
FAILED solution/riceintensification/tests/test_riceintensification.py::test_deep_results - AssertionError: 
FAILED solution/ships/tests/test_ships.py::test_key_results - AssertionError: 
FAILED solution/ships/tests/test_ships.py::test_deep_results - AssertionError: 
FAILED solution/solarhotwater/tests/test_solarhotwater.py::test_key_results - AssertionError: 
FAILED solution/solarhotwater/tests/test_solarhotwater.py::test_deep_results - AssertionError: 
FAILED solution/trains/tests/test_trains.py::test_loader - zipfile.BadZipFile: File is not a zip file
FAILED solution/trains/tests/test_trains.py::test_key_results - zipfile.BadZipFile: File is not a zip file
FAILED solution/trains/tests/test_trains.py::test_deep_results - zipfile.BadZipFile: File is not a zip file
FAILED solution/wastetoenergy/tests/test_wastetoenergy.py::test_deep_results - AssertionError: 
FAILED solution/waterefficiency/tests/test_waterefficiency.py::test_deep_results - AssertionError: 
================================================================== 28 failed, 1322 passed, 12 skipped, 84 warnings in 731.11s (0:12:11) ===================================================================
denised commented 2 years ago

Thank you @gerald-scharitzer ! I will try to spend some time in the near future to incorporate this functionality into some introductory examples and expand its use accordingly.