PSLmodels / OG-USA

Overlapping-generations macroeconomic model for evaluating fiscal policy in the United States
https://pslmodels.github.io/OG-USA/
Creative Commons Zero v1.0 Universal
19 stars 34 forks source link

Remove demographics #86

Closed jdebacker closed 5 months ago

jdebacker commented 5 months ago

This PR updates calibrate.py to use ogcore.demographics and removes the demographics module and tests from OG-USA.

jdebacker commented 5 months ago

Note, these tests will not pass until we release OG-Core 0.11.0 on PyPI.

jdebacker commented 5 months ago

@rickecon This PR is ready for review.

codecov-commenter commented 5 months ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (1982230) 73.76% compared to head (dcbe46b) 70.82%.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## master #86 +/- ## ========================================== - Coverage 73.76% 70.82% -2.95% ========================================== Files 24 22 -2 Lines 1616 1340 -276 ========================================== - Hits 1192 949 -243 + Misses 424 391 -33 ``` | [Flag](https://app.codecov.io/gh/PSLmodels/OG-USA/pull/86/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=PSLmodels) | Coverage Δ | | |---|---|---| | [unittests](https://app.codecov.io/gh/PSLmodels/OG-USA/pull/86/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=PSLmodels) | `70.82% <100.00%> (-2.95%)` | :arrow_down: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=PSLmodels#carryforward-flags-in-the-pull-request-comment) to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

rickecon commented 5 months ago

@jdebacker. All GH Action CI tests pass now. But I have a few failures of tests that run only locally. I get 2 failures in the test_get_micro_data.py module, 3 failures in test_psid_data_setup.py.

(ogusa-dev) richardevans@Richards-MacBook-Pro-2 OG-USA % pytest ./tests/
/opt/anaconda3/envs/ogusa-dev/lib/python3.12/site-packages/pep8.py:110: FutureWarning: Possible nested set at position 1
  EXTRANEOUS_WHITESPACE_REGEX = re.compile(r'[[({] | []}),;:]')
=========================================== test session starts ===========================================
platform darwin -- Python 3.12.1, pytest-8.0.0, pluggy-1.4.0
rootdir: /Users/richardevans/Docs/Economics/OSE/OG-USA
configfile: pytest.ini
plugins: xdist-3.5.0, pep8-1.0.6
collected 37 items                                                                                        

tests/test_calibrate.py ....                                                                        [ 10%]
tests/test_get_micro_data.py ........FF....                                                         [ 48%]
tests/test_income.py ............                                                                   [ 81%]
tests/test_psid_data_setup.py FFF                                                                   [ 89%]
tests/test_run_example.py .                                                                         [ 91%]
tests/test_utils.py .                                                                               [ 94%]
tests/test_wealth.py ..                                                                             [100%]
...
========================================= short test summary info =========================================
FAILED tests/test_get_micro_data.py::test_get_calculator_puf - ValueError: could not read /Users/richardevans/Docs/Economics/OSE/OG-USA/tests/../puf.csv data from egg
FAILED tests/test_get_micro_data.py::test_get_calculator_puf_from_file - ValueError: could not read /Users/richardevans/Docs/Economics/OSE/OG-USA/tests/../puf.csv data from egg
FAILED tests/test_psid_data_setup.py::test_prep_data - pandas.errors.InvalidIndexError: slice(None, None, None)
FAILED tests/test_psid_data_setup.py::test_num_obs_total - pandas.errors.InvalidIndexError: slice(None, None, None)
FAILED tests/test_psid_data_setup.py::test_indicator_vars - pandas.errors.InvalidIndexError: slice(None, None, None)
========================= 5 failed, 32 passed, 114 warnings in 1330.49s (0:22:10) =========================

The two errors in test_get_micro_data.py are coming from me not having the puf.csv in that directory. The three errors in test_psid_data_setup.py are pandas object slicing error. I should be able to fix the first two by placing the puf.csv in the right file. And I should be able to correct the last three by fixing the slicing for the updated objects.

rickecon commented 5 months ago

Adding the puf.csv fixed the two failing tests in test_get_micro_data.py.

rickecon commented 5 months ago

@jdebacker. The three slicing errors are persisting in my local tests of test_psid_data_setup.py. The first of those tests test_prep_data() is one that is passing in the GH Action CI.

I think the cause of this might be that my new ogusa-dev conda environment has Python version 3.12.1. The ogcore-dev conda environment.yml restricts python>=3.7.7, <3.12 the the OG-Core setup.py, which is what pip installs only requires python>=3.7.7.

The OG-USA environment.yml only requires that python>=3.7.7, as well as the setup.py. We might need to restrict Python<3.12 in both repositories.

I am going ahead and merging this.