PSLmodels / OG-USA

Overlapping-generations macroeconomic model for evaluating fiscal policy in the United States
https://pslmodels.github.io/OG-USA/
Creative Commons Zero v1.0 Universal
19 stars 34 forks source link

Update handling of tax func params for new ogcore #69

Closed jdebacker closed 1 year ago

jdebacker commented 1 year ago

This PR updates calibrate.py to handle tax function parameters that are in lists, as they are in OG-Core v 0.10.1.

jdebacker commented 1 year ago

@rickecon This PR should be ready to go (all tests pass on my local machine).

But the tests are failing here because:

E   ModuleNotFoundError: No module named 'pandas'

Despite pandas clearly being a dependency listed in the environment.yml file.

Ideas?

jdebacker commented 1 year ago

I have included pandas in the environment.yml file and in setup.py -- but still getting error that pandas not installed on GH Action.

Ideas @rickecon @nikhilwoodruff ?

codecov-commenter commented 1 year ago

Codecov Report

Patch coverage: 10.86% and project coverage change: -0.20 :warning:

Comparison is base (c681306) 78.56% compared to head (8cae181) 78.36%.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## master #69 +/- ## ========================================== - Coverage 78.56% 78.36% -0.20% ========================================== Files 21 21 Lines 1376 1382 +6 ========================================== + Hits 1081 1083 +2 - Misses 295 299 +4 ``` | Flag | Coverage Δ | | |---|---|---| | unittests | `78.36% <10.86%> (-0.20%)` | :arrow_down: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=PSLmodels#carryforward-flags-in-the-pull-request-comment) to find out more. | [Impacted Files](https://codecov.io/gh/PSLmodels/OG-USA/pull/69?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=PSLmodels) | Coverage Δ | | |---|---|---| | [ogusa/bequest\_transmission.py](https://codecov.io/gh/PSLmodels/OG-USA/pull/69?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=PSLmodels#diff-b2d1c2EvYmVxdWVzdF90cmFuc21pc3Npb24ucHk=) | `68.35% <0.00%> (ø)` | | | [ogusa/tests/test\_get\_micro\_data.py](https://codecov.io/gh/PSLmodels/OG-USA/pull/69?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=PSLmodels#diff-b2d1c2EvdGVzdHMvdGVzdF9nZXRfbWljcm9fZGF0YS5weQ==) | `81.48% <0.00%> (ø)` | | | [setup.py](https://codecov.io/gh/PSLmodels/OG-USA/pull/69?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=PSLmodels#diff-c2V0dXAucHk=) | `0.00% <ø> (ø)` | | | [ogusa/calibrate.py](https://codecov.io/gh/PSLmodels/OG-USA/pull/69?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=PSLmodels#diff-b2d1c2EvY2FsaWJyYXRlLnB5) | `38.51% <2.50%> (-0.72%)` | :arrow_down: | | [ogusa/tests/test\_calibrate.py](https://codecov.io/gh/PSLmodels/OG-USA/pull/69?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=PSLmodels#diff-b2d1c2EvdGVzdHMvdGVzdF9jYWxpYnJhdGUucHk=) | `100.00% <100.00%> (ø)` | | | [ogusa/transfer\_distribution.py](https://codecov.io/gh/PSLmodels/OG-USA/pull/69?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=PSLmodels#diff-b2d1c2EvdHJhbnNmZXJfZGlzdHJpYnV0aW9uLnB5) | `84.21% <100.00%> (ø)` | | | [ogusa/utils.py](https://codecov.io/gh/PSLmodels/OG-USA/pull/69?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=PSLmodels#diff-b2d1c2EvdXRpbHMucHk=) | `100.00% <100.00%> (ø)` | | | [ogusa/wealth.py](https://codecov.io/gh/PSLmodels/OG-USA/pull/69?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=PSLmodels#diff-b2d1c2Evd2VhbHRoLnB5) | `77.14% <100.00%> (ø)` | | Help us with your feedback. Take ten seconds to tell us [how you rate us](https://about.codecov.io/nps?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=PSLmodels). Have a feature suggestion? [Share it here.](https://app.codecov.io/gh/feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=PSLmodels)

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.

jdebacker commented 1 year ago

@rickecon Changes now avoid miniconda build issue and tests are passing on 3.8 on GH Actions. Not sure why there are failures on 3.7 and 3.9, but all tests pass locally on Python 3.10 for me. If you can confirm that, I think this PR is good to go.

rickecon commented 1 year ago

@jdebacker. All CI tests are passing. And I updated my repo with your newest changes, updated my conda environment, and re-ran all the tests locally. I got the following one error.

========== test session starts ===========
platform darwin -- Python 3.10.10, pytest-7.3.0, pluggy-1.0.0
rootdir: /Users/richardevans/Docs/Economics/OSE/OG-USA
configfile: pytest.ini
plugins: xdist-3.2.1, pep8-1.0.6
collected 45 items                                                                                               

ogusa/tests/test_calibrate.py ....                  [  8%]
ogusa/tests/test_demographics.py ........           [ 26%]
ogusa/tests/test_get_micro_data.py ..F...........   [ 57%]
ogusa/tests/test_income.py ............             [ 84%]
ogusa/tests/test_psid_data_setup.py ...             [ 91%]
ogusa/tests/test_run_example.py .                   [ 93%]
ogusa/tests/test_utils.py .                         [ 95%]
ogusa/tests/test_wealth.py ..                       [100%]
...
_________________________________________________ test_puf_path __________________________________________________

    def test_puf_path():
        """
        Check that setting `data` to None uses the puf file
        """
        baseline = False
        start_year = 2016
        reform = {"II_em": {2017: 10000}}

        # puf.csv in ogusa/
        if os.path.exists(PUF_PATH):
            calc = get_micro_data.get_calculator(
                baseline, start_year, reform=reform, data=PUF_PATH
            )
            # blind_head is only in the CPS file and e00700 is only in the
            # PUF.  See taxcalc/records_variables.json
>           assert (
                calc.array("blind_head").sum() == 0
                and calc.array("e00700").sum() > 0
            )
E           AssertionError: assert (3115 == 0)
E            +  where 3115 = <built-in method sum of numpy.ndarray object at 0x19d8cc390>()
E            +    where <built-in method sum of numpy.ndarray object at 0x19d8cc390> = array([0, 0, 0, ..., 0, 0, 0], dtype=int32).sum
E            +      where array([0, 0, 0, ..., 0, 0, 0], dtype=int32) = <bound method Calculator.array of <taxcalc.calculator.Calculator object at 0x18c113f70>>('blind_head')
E            +        where <bound method Calculator.array of <taxcalc.calculator.Calculator object at 0x18c113f70>> = <taxcalc.calculator.Calculator object at 0x18c113f70>.array

ogusa/tests/test_get_micro_data.py:88: AssertionError
...
========= short test summary info ==========
FAILED ogusa/tests/test_get_micro_data.py::test_puf_path - AssertionError: assert (3115 == 0)
======= 1 failed, 44 passed, 66 warnings in 2031.47s (0:33:51) ===========

This looks like a weird error between Tax-Calculator and Numpy. Let me know if you have any ideas.

rickecon commented 1 year ago

@jdebacker. Here is my conda list of my new ogusa-dev environment.

(ogusa-dev) richardevans@Richards-MacBook-Pro OG-USA % conda list
# packages in environment at /opt/anaconda3/envs/ogusa-dev:
#
# Name                    Version                   Build  Channel
_r-mutex                  1.0.1               anacondar_1    conda-forge
accessible-pygments       0.0.4              pyhd8ed1ab_0    conda-forge
aiohttp                   3.8.4           py310h90acd4f_0    conda-forge
aiosignal                 1.3.1              pyhd8ed1ab_0    conda-forge
alabaster                 0.7.13             pyhd8ed1ab_0    conda-forge
appnope                   0.1.3                    pypi_0    pypi
arrow-cpp                 11.0.0          h694c41f_13_cpu    conda-forge
astor                     0.8.1              pyh9f0ad1d_0    conda-forge
astroid                   2.15.2          py310h2ec42d9_0    conda-forge
asttokens                 2.2.1                    pypi_0    pypi
async-timeout             4.0.2              pyhd8ed1ab_0    conda-forge
attrs                     22.2.0             pyh71513ae_0    conda-forge
aws-c-auth                0.6.26               hcaaecf5_1    conda-forge
aws-c-cal                 0.5.21               hf54dd2f_2    conda-forge
aws-c-common              0.8.14               hb7f2c08_0    conda-forge
aws-c-compression         0.2.16               h99c63db_5    conda-forge
aws-c-event-stream        0.2.20               h0ae4c0f_4    conda-forge
aws-c-http                0.7.6                hcdd7835_0    conda-forge
aws-c-io                  0.13.19              had634fe_3    conda-forge
aws-c-mqtt                0.8.6               ha4e05e3_12    conda-forge
aws-c-s3                  0.2.7                hba8c00c_1    conda-forge
aws-c-sdkutils            0.1.8                h99c63db_0    conda-forge
aws-checksums             0.1.14               h99c63db_5    conda-forge
aws-crt-cpp               0.19.8              h8b21dc2_12    conda-forge
aws-sdk-cpp               1.10.57              h0d082ec_8    conda-forge
babel                     2.12.1             pyhd8ed1ab_1    conda-forge
backcall                  0.2.0                    pypi_0    pypi
beautifulsoup4            4.12.2             pyha770c72_0    conda-forge
black                     23.3.0          py310h2ec42d9_0    conda-forge
bokeh                     2.4.3              pyhd8ed1ab_3    conda-forge
brotli                    1.0.9                hb7f2c08_8    conda-forge
brotli-bin                1.0.9                hb7f2c08_8    conda-forge
brotlipy                  0.7.0           py310h90acd4f_1005    conda-forge
bwidget                   1.9.14               h694c41f_1    conda-forge
bzip2                     1.0.8                h0d85af4_4    conda-forge
c-ares                    1.18.1               h0d85af4_0    conda-forge
ca-certificates           2022.12.7            h033912b_0    conda-forge
cached_property           1.5.2              pyha770c72_1    conda-forge
cachetools                5.3.0                    pypi_0    pypi
cairo                     1.16.0            h297c08e_1015    conda-forge
cctools_osx-64            973.0.1             h5676edf_13    conda-forge
certifi                   2022.12.7          pyhd8ed1ab_0    conda-forge
cffi                      1.15.1          py310ha78151a_3    conda-forge
charset-normalizer        2.1.1              pyhd8ed1ab_0    conda-forge
clang                     16.0.1               h49458c0_0    conda-forge
clang-16                  16.0.1          default_hd25f202_0    conda-forge
clang_osx-64              16.0.1               hac0a0c4_0    conda-forge
clangxx                   16.0.1          default_hd25f202_0    conda-forge
clangxx_osx-64            16.0.1               h950475c_0    conda-forge
click                     8.1.3           unix_pyhd8ed1ab_2    conda-forge
cloudpickle               2.2.1              pyhd8ed1ab_0    conda-forge
colorama                  0.4.6              pyhd8ed1ab_0    conda-forge
comm                      0.1.3                    pypi_0    pypi
compiler-rt               16.0.1               h613da45_0    conda-forge
compiler-rt_osx-64        16.0.1               h613da45_0    conda-forge
contourpy                 1.0.7           py310ha23aa8a_0    conda-forge
coverage                  7.2.3           py310h90acd4f_0    conda-forge
cryptography              40.0.1          py310hdd0c95c_0    conda-forge
cs-kit                    1.16.9                   pypi_0    pypi
cs-storage                1.11.1                   pypi_0    pypi
cs2tc                     0.1.1                    pypi_0    pypi
curl                      7.88.1               h6df9250_1    conda-forge
cycler                    0.11.0             pyhd8ed1ab_0    conda-forge
cython                    0.29.34         py310h7a76584_0    conda-forge
cytoolz                   0.12.0          py310h90acd4f_1    conda-forge
dask                      2023.3.2           pyhd8ed1ab_0    conda-forge
dask-core                 2023.3.2           pyhd8ed1ab_0    conda-forge
debugpy                   1.6.7                    pypi_0    pypi
decorator                 5.1.1                    pypi_0    pypi
dill                      0.3.6              pyhd8ed1ab_1    conda-forge
distributed               2023.3.2.1         pyhd8ed1ab_0    conda-forge
docutils                  0.18.1                   pypi_0    pypi
et-xmlfile                1.1.0                    pypi_0    pypi
exceptiongroup            1.1.1              pyhd8ed1ab_0    conda-forge
execnet                   1.9.0              pyhd8ed1ab_0    conda-forge
executing                 1.2.0                    pypi_0    pypi
expat                     2.5.0                hf0c8a7f_1    conda-forge
fastjsonschema            2.16.3                   pypi_0    pypi
font-ttf-dejavu-sans-mono 2.37                 hab24e00_0    conda-forge
font-ttf-inconsolata      3.000                h77eed37_0    conda-forge
font-ttf-source-code-pro  2.038                h77eed37_0    conda-forge
font-ttf-ubuntu           0.83                 hab24e00_0    conda-forge
fontconfig                2.14.2               h5bb23bf_0    conda-forge
fonts-conda-ecosystem     1                             0    conda-forge
fonts-conda-forge         1                             0    conda-forge
fonttools                 4.39.3          py310h90acd4f_0    conda-forge
formulaic                 0.3.4                    pypi_0    pypi
freetype                  2.12.1               h3f81eb7_1    conda-forge
fribidi                   1.0.10               hbcb3906_0    conda-forge
frozenlist                1.3.3           py310h90acd4f_0    conda-forge
fsspec                    2023.3.0                 pypi_0    pypi
gcsfs                     2023.3.0                 pypi_0    pypi
gettext                   0.21.1               h8a4c099_0    conda-forge
gflags                    2.2.2             hb1e8313_1004    conda-forge
gfortran_impl_osx-64      11.3.0              h1f927f5_31    conda-forge
gfortran_osx-64           11.3.0               h18f7dce_1    conda-forge
glog                      0.6.0                h8ac2a54_0    conda-forge
gmp                       6.2.1                h2e338ed_0    conda-forge
google-api-core           2.11.0                   pypi_0    pypi
google-auth               2.17.2                   pypi_0    pypi
google-auth-oauthlib      1.0.0                    pypi_0    pypi
google-cloud-core         2.3.2                    pypi_0    pypi
google-cloud-storage      2.8.0                    pypi_0    pypi
google-crc32c             1.5.0                    pypi_0    pypi
google-resumable-media    2.4.1                    pypi_0    pypi
googleapis-common-protos  1.59.0                   pypi_0    pypi
graphite2                 1.3.13            h2e338ed_1001    conda-forge
graphlib-backport         1.0.3              pyhd8ed1ab_0    conda-forge
greenlet                  2.0.2                    pypi_0    pypi
gsl                       2.7                  h93259b0_0    conda-forge
harfbuzz                  6.0.0                hc28939c_1    conda-forge
heapdict                  1.0.1                      py_0    conda-forge
icu                       72.1                 h7336db1_0    conda-forge
idna                      3.4                pyhd8ed1ab_0    conda-forge
imagesize                 1.4.1              pyhd8ed1ab_0    conda-forge
importlib-metadata        6.3.0              pyha770c72_0    conda-forge
importlib_metadata        6.3.0                hd8ed1ab_0    conda-forge
iniconfig                 2.0.0              pyhd8ed1ab_0    conda-forge
interface_meta            1.3.0              pyhd8ed1ab_0    conda-forge
ipykernel                 6.22.0                   pypi_0    pypi
ipython                   8.12.0                   pypi_0    pypi
isl                       0.25                 hb486fe8_0    conda-forge
isort                     5.12.0             pyhd8ed1ab_1    conda-forge
jedi                      0.18.2                   pypi_0    pypi
jinja2                    3.1.2              pyhd8ed1ab_1    conda-forge
jsonschema                4.17.3                   pypi_0    pypi
jupyter-book              0.15.1                   pypi_0    pypi
jupyter-cache             0.5.0                    pypi_0    pypi
jupyter-client            8.1.0                    pypi_0    pypi
jupyter-core              5.3.0                    pypi_0    pypi
kiwisolver                1.4.4           py310ha23aa8a_1    conda-forge
krb5                      1.20.1               h049b76e_0    conda-forge
latexcodec                2.0.1                    pypi_0    pypi
lazy-object-proxy         1.9.0           py310h90acd4f_0    conda-forge
lcms2                     2.15                 h2dcdeff_1    conda-forge
ld64_osx-64               609                 hbfe4790_13    conda-forge
lerc                      4.0.0                hb486fe8_0    conda-forge
libabseil                 20230125.0      cxx17_hf0c8a7f_1    conda-forge
libarrow                  11.0.0          h547aefa_13_cpu    conda-forge
libblas                   3.9.0           16_osx64_openblas    conda-forge
libbrotlicommon           1.0.9                hb7f2c08_8    conda-forge
libbrotlidec              1.0.9                hb7f2c08_8    conda-forge
libbrotlienc              1.0.9                hb7f2c08_8    conda-forge
libcblas                  3.9.0           16_osx64_openblas    conda-forge
libclang-cpp16            16.0.1          default_hd25f202_0    conda-forge
libcrc32c                 1.1.2                he49afe7_0    conda-forge
libcurl                   7.88.1               h6df9250_1    conda-forge
libcxx                    16.0.1               h71dddab_0    conda-forge
libdeflate                1.18                 hac1461d_0    conda-forge
libedit                   3.1.20191231         h0678c8f_2    conda-forge
libev                     4.33                 haf1e3a3_1    conda-forge
libevent                  2.1.10               h7d65743_4    conda-forge
libexpat                  2.5.0                hf0c8a7f_1    conda-forge
libffi                    3.4.2                h0d85af4_5    conda-forge
libgfortran               5.0.0           11_3_0_h97931a8_31    conda-forge
libgfortran-devel_osx-64  11.3.0              h824d247_31    conda-forge
libgfortran5              12.2.0              he409387_31    conda-forge
libglib                   2.74.1               h4c723e1_1    conda-forge
libgoogle-cloud           2.8.0                h176059f_1    conda-forge
libgrpc                   1.52.1               h5bc3d57_1    conda-forge
libiconv                  1.17                 hac89ed1_0    conda-forge
libjpeg-turbo             2.1.5.1              hb7f2c08_0    conda-forge
liblapack                 3.9.0           16_osx64_openblas    conda-forge
libllvm11                 11.1.0               h8fb7429_5    conda-forge
libllvm16                 16.0.1               h7001e86_0    conda-forge
libnghttp2                1.52.0               he2ab024_0    conda-forge
libopenblas               0.3.21          openmp_h429af6e_3    conda-forge
libpng                    1.6.39               ha978bb4_0    conda-forge
libprotobuf               3.21.12              hbc0c0cd_0    conda-forge
libsqlite                 3.40.0               ha978bb4_0    conda-forge
libssh2                   1.10.0               h47af595_3    conda-forge
libthrift                 0.18.1               h16802d8_0    conda-forge
libtiff                   4.5.0                hedf67fa_6    conda-forge
libutf8proc               2.8.0                hb7f2c08_0    conda-forge
libwebp-base              1.3.0                hb7f2c08_0    conda-forge
libxcb                    1.13              h0d85af4_1004    conda-forge
libxml2                   2.10.3               h554bb67_6    conda-forge
libzlib                   1.2.13               hfd90126_4    conda-forge
linearmodels              4.27            py310h936d966_2    conda-forge
linkify-it-py             2.0.0                    pypi_0    pypi
llvm-openmp               16.0.1               h61d9ccf_0    conda-forge
llvm-tools                16.0.1               h7001e86_0    conda-forge
llvmlite                  0.39.1          py310h2bfb868_1    conda-forge
locket                    1.0.0              pyhd8ed1ab_0    conda-forge
lxml                      4.9.2                    pypi_0    pypi
lz4                       4.3.2           py310h8d4e1d9_0    conda-forge
lz4-c                     1.9.4                hf0c8a7f_0    conda-forge
make                      4.3                  h22f3db7_1    conda-forge
markdown-it-py            2.2.0                    pypi_0    pypi
markupsafe                2.1.2           py310h90acd4f_0    conda-forge
marshmallow               3.19.0             pyhd8ed1ab_0    conda-forge
matplotlib                3.7.1           py310h2ec42d9_0    conda-forge
matplotlib-base           3.7.1           py310he725631_0    conda-forge
matplotlib-inline         0.1.6                    pypi_0    pypi
mccabe                    0.7.0              pyhd8ed1ab_0    conda-forge
mdit-py-plugins           0.3.5                    pypi_0    pypi
mdurl                     0.1.2                    pypi_0    pypi
mpc                       1.3.1                h81bd1dd_0    conda-forge
mpfr                      4.2.0                h4f9bd69_0    conda-forge
msgpack-python            1.0.5           py310ha23aa8a_0    conda-forge
multidict                 6.0.4           py310h90acd4f_0    conda-forge
munkres                   1.1.4              pyh9f0ad1d_0    conda-forge
mypy_extensions           1.0.0              pyha770c72_0    conda-forge
myst-nb                   0.17.1                   pypi_0    pypi
myst-parser               0.18.1                   pypi_0    pypi
nbclient                  0.5.13                   pypi_0    pypi
nbformat                  5.8.0                    pypi_0    pypi
ncurses                   6.3                  h96cf925_1    conda-forge
nest-asyncio              1.5.6                    pypi_0    pypi
numba                     0.56.4          py310h62db5c2_1    conda-forge
numpy                     1.23.5          py310h1b7c290_0    conda-forge
oauthlib                  3.2.2                    pypi_0    pypi
ogcore                    0.10.7                   pypi_0    pypi
ogusa                     0.0.0                    pypi_0    pypi
openjpeg                  2.5.0                h13ac156_2    conda-forge
openpyxl                  3.1.2                    pypi_0    pypi
openssl                   3.1.0                hfd90126_0    conda-forge
orc                       1.8.3                ha9d861c_0    conda-forge
packaging                 23.0               pyhd8ed1ab_0    conda-forge
pandas                    2.0.0           py310hecf8f37_0    conda-forge
pandas-datareader         0.10.0                   pypi_0    pypi
pango                     1.50.14              hbd9bf65_0    conda-forge
paramtools                0.18.1                   pypi_0    pypi
parquet-cpp               1.5.1                         2    conda-forge
parso                     0.8.3                    pypi_0    pypi
partd                     1.3.0              pyhd8ed1ab_0    conda-forge
pathspec                  0.11.1             pyhd8ed1ab_0    conda-forge
patsy                     0.5.3              pyhd8ed1ab_0    conda-forge
pcre2                     10.40                h1c4e4bc_0    conda-forge
pep8                      1.7.1                      py_0    conda-forge
pexpect                   4.8.0                    pypi_0    pypi
pickleshare               0.7.5                    pypi_0    pypi
pillow                    9.5.0           py310h6b2f720_0    conda-forge
pip                       23.0.1             pyhd8ed1ab_0    conda-forge
pixman                    0.40.0               hbcb3906_0    conda-forge
platformdirs              3.2.0              pyhd8ed1ab_0    conda-forge
pluggy                    1.0.0              pyhd8ed1ab_5    conda-forge
pooch                     1.7.0              pyha770c72_3    conda-forge
prompt-toolkit            3.0.38                   pypi_0    pypi
property-cached           1.6.4                      py_0    conda-forge
property_cached           1.6.4                         0    conda-forge
protobuf                  4.22.1                   pypi_0    pypi
psutil                    5.9.4           py310h90acd4f_0    conda-forge
pthread-stubs             0.4               hc929b4f_1001    conda-forge
ptyprocess                0.7.0                    pypi_0    pypi
pure-eval                 0.2.2                    pypi_0    pypi
pyarrow                   11.0.0          py310h435aefc_13_cpu    conda-forge
pyasn1                    0.4.8                    pypi_0    pypi
pyasn1-modules            0.2.8                    pypi_0    pypi
pybtex                    0.24.0                   pypi_0    pypi
pybtex-docutils           1.0.2                    pypi_0    pypi
pycodestyle               2.10.0             pyhd8ed1ab_0    conda-forge
pycparser                 2.21               pyhd8ed1ab_0    conda-forge
pydata-sphinx-theme       0.13.3             pyhd8ed1ab_0    conda-forge
pygments                  2.15.0             pyhd8ed1ab_0    conda-forge
pyhdfe                    0.1.1              pyhd8ed1ab_0    conda-forge
pylint                    2.17.2             pyhd8ed1ab_0    conda-forge
pyopenssl                 23.1.1             pyhd8ed1ab_0    conda-forge
pyparsing                 3.0.9              pyhd8ed1ab_0    conda-forge
pyrsistent                0.19.3                   pypi_0    pypi
pysocks                   1.7.1              pyha2e5f31_6    conda-forge
pytest                    7.3.0              pyhd8ed1ab_0    conda-forge
pytest-cache              1.0                        py_2    conda-forge
pytest-pep8               1.0.6                      py_2    conda-forge
pytest-xdist              3.2.1              pyhd8ed1ab_0    conda-forge
python                    3.10.10         he7542f4_0_cpython    conda-forge
python-dateutil           2.8.2              pyhd8ed1ab_0    conda-forge
python-tzdata             2023.3             pyhd8ed1ab_0    conda-forge
python_abi                3.10                    3_cp310    conda-forge
pytz                      2023.3             pyhd8ed1ab_0    conda-forge
pytz-deprecation-shim     0.1.0.post0     py310h2ec42d9_3    conda-forge
pyyaml                    6.0             py310h90acd4f_5    conda-forge
pyzmq                     25.0.2                   pypi_0    pypi
r-base                    4.2.3                h29e1f52_2    conda-forge
re2                       2023.02.02           hf0c8a7f_0    conda-forge
readline                  8.2                  h9e318b2_1    conda-forge
requests                  2.28.2             pyhd8ed1ab_1    conda-forge
requests-oauthlib         1.3.1                    pypi_0    pypi
rpy2                      3.5.10          py310r42h936d966_0    conda-forge
rsa                       4.9                      pypi_0    pypi
scipy                     1.10.1          py310h240c617_0    conda-forge
setuptools                67.6.1             pyhd8ed1ab_0    conda-forge
setuptools-scm            6.4.2                    pypi_0    pypi
sigtool                   0.1.3                h88f4db0_0    conda-forge
simplegeneric             0.8.1                      py_1    conda-forge
six                       1.16.0             pyh6c4a22f_0    conda-forge
snappy                    1.1.10               h225ccf5_0    conda-forge
snowballstemmer           2.2.0              pyhd8ed1ab_0    conda-forge
sortedcontainers          2.4.0              pyhd8ed1ab_0    conda-forge
soupsieve                 2.3.2.post1        pyhd8ed1ab_0    conda-forge
sphinx                    5.0.2                    pypi_0    pypi
sphinx-book-theme         1.0.1              pyhd8ed1ab_0    conda-forge
sphinx-comments           0.0.3                    pypi_0    pypi
sphinx-copybutton         0.5.1                    pypi_0    pypi
sphinx-design             0.3.0                    pypi_0    pypi
sphinx-external-toc       0.3.1                    pypi_0    pypi
sphinx-jupyterbook-latex  0.5.2                    pypi_0    pypi
sphinx-multitoc-numbering 0.1.3                    pypi_0    pypi
sphinx-thebe              0.2.1                    pypi_0    pypi
sphinx-togglebutton       0.3.2                    pypi_0    pypi
sphinxcontrib-applehelp   1.0.4              pyhd8ed1ab_0    conda-forge
sphinxcontrib-bibtex      2.5.0                    pypi_0    pypi
sphinxcontrib-devhelp     1.0.2                      py_0    conda-forge
sphinxcontrib-htmlhelp    2.0.1              pyhd8ed1ab_0    conda-forge
sphinxcontrib-jsmath      1.0.1                      py_0    conda-forge
sphinxcontrib-qthelp      1.0.3                      py_0    conda-forge
sphinxcontrib-serializinghtml 1.1.5              pyhd8ed1ab_2    conda-forge
sqlalchemy                1.4.47                   pypi_0    pypi
stack-data                0.6.2                    pypi_0    pypi
statsmodels               0.13.5          py310h936d966_2    conda-forge
tabulate                  0.9.0                    pypi_0    pypi
tapi                      1100.0.11            h9ce4665_0    conda-forge
taxcalc                   3.3.0              pyhd8ed1ab_0    conda-forge
tblib                     1.7.0              pyhd8ed1ab_0    conda-forge
tk                        8.6.12               h5dbffcc_0    conda-forge
tktable                   2.10                 h49f0cf7_3    conda-forge
tomli                     2.0.1              pyhd8ed1ab_0    conda-forge
tomlkit                   0.11.7             pyha770c72_0    conda-forge
toolz                     0.12.0             pyhd8ed1ab_0    conda-forge
tornado                   6.2             py310h90acd4f_1    conda-forge
traitlets                 5.9.0                    pypi_0    pypi
typing                    3.10.0.0           pyhd8ed1ab_0    conda-forge
typing-extensions         4.5.0                hd8ed1ab_0    conda-forge
typing_extensions         4.5.0              pyha770c72_0    conda-forge
tzdata                    2023c                h71feb2d_0    conda-forge
tzlocal                   4.3             py310h2ec42d9_0    conda-forge
uc-micro-py               1.0.1                    pypi_0    pypi
unicodedata2              15.0.0          py310h90acd4f_0    conda-forge
urllib3                   1.26.15            pyhd8ed1ab_0    conda-forge
wcwidth                   0.2.6                    pypi_0    pypi
wheel                     0.40.0             pyhd8ed1ab_0    conda-forge
wrapt                     1.15.0          py310h90acd4f_0    conda-forge
xlwt                      1.3.0                      py_1    conda-forge
xorg-libxau               1.0.9                h35c211d_0    conda-forge
xorg-libxdmcp             1.1.3                h35c211d_0    conda-forge
xz                        5.2.6                h775f41a_0    conda-forge
yaml                      0.2.5                h0d85af4_2    conda-forge
yarl                      1.8.2           py310h90acd4f_0    conda-forge
zict                      2.2.0              pyhd8ed1ab_0    conda-forge
zipp                      3.15.0             pyhd8ed1ab_0    conda-forge
zlib                      1.2.13               hfd90126_4    conda-forge
zstd                      1.5.2                hbc0c0cd_6    conda-forge
jdebacker commented 1 year ago

@rickecon The error you are seeing is due to using a different version of the puf.csv file that is created from the TaxData project. Older files, such as the one I am testing against, do not have the blind_head variable, therefore it's set to zero for each record. This is what the test asserts (because the sum of blind_head should be zero for the older PUF, but positive for the CPS based file from TaxData).

However, recent TaxData impute the blind_head variable (or get it from a more recent PUF, I'm not sure), so you are seeing a test failure with that more recent PUF file.

In light of this, and assuming we want to test that the puf.csv file loads, perhaps we can just assert that no error is raised when trying to load the PUF file?

rickecon commented 1 year ago

@jdebacker. I just downloaded the most recent puf_2022.csv file. I am happy to merge this if there is another PUF file that passes this test. Let me know what you want me to do here.

jdebacker commented 1 year ago

@rickecon I modified the test to work for any vintage puf by just confirming that a variable that is in the PUF based file, but not the CPS file is nonzero.

I believe this PR can be merged.

rickecon commented 1 year ago

@jdebacker. Thanks for this. Looks great. Merging.