PSLmodels / Tax-Brain

Tax-Brain is an integrator model for PSL tax models
http://taxbrain.pslmodels.org/
MIT License
8 stars 12 forks source link

Failed C/S test #107

Closed hdoupe closed 4 years ago

hdoupe commented 4 years ago

I got a test failure when running Tax-Brain's compute studio tests with the new version of Tax-Calculator:

(taxbrain-dev) hankdoupe@system76-pc:~/Documents/Tax-Brain$ py.test cs-config/ -v
==================================================================================== test session starts =====================================================================================
platform linux -- Python 3.7.3, pytest-5.3.1, py-1.8.0, pluggy-0.13.0 -- /home/hankdoupe/miniconda3/envs/taxbrain-dev/bin/python
cachedir: .pytest_cache
rootdir: /home/hankdoupe/Documents/Tax-Brain/cs-config
collected 12 items                                                                                                                                                                           

cs-config/cs_config/tests/test_functions.py::TestFunctions1::test_all_data_specified PASSED                                                                                            [  8%]
cs-config/cs_config/tests/test_functions.py::TestFunctions1::test_get_version PASSED                                                                                                   [ 16%]
cs-config/cs_config/tests/test_functions.py::TestFunctions1::test_get_inputs PASSED                                                                                                    [ 25%]
cs-config/cs_config/tests/test_functions.py::TestFunctions1::test_validate_inputs PASSED                                                                                               [ 33%]
cs-config/cs_config/tests/test_functions.py::TestFunctions1::test_run_model PASSED                                                                                                     [ 41%]
cs-config/cs_config/tests/test_functions.py::TestFunctions2::test_all_data_specified PASSED                                                                                            [ 50%]
cs-config/cs_config/tests/test_functions.py::TestFunctions2::test_get_version PASSED                                                                                                   [ 58%]
cs-config/cs_config/tests/test_functions.py::TestFunctions2::test_get_inputs PASSED                                                                                                    [ 66%]
cs-config/cs_config/tests/test_functions.py::TestFunctions2::test_validate_inputs PASSED                                                                                               [ 75%]
cs-config/cs_config/tests/test_functions.py::TestFunctions2::test_run_model PASSED                                                                                                     [ 83%]
cs-config/cs_config/tests/test_functions.py::test_convert_adj FAILED                                                                                                                   [ 91%]
cs-config/cs_config/tests/test_functions.py::test_convert_adj_w_index PASSED                                                                                                           [100%]

========================================================================================== FAILURES ==========================================================================================
______________________________________________________________________________________ test_convert_adj ______________________________________________________________________________________

    def test_convert_adj():
        adj = {
            "STD": [
                {"MARS": "single", "year": 2019, "value": 0},
                {"MARS": "mjoint", "year": 2019, "value": 1},
                {"MARS": "mjoint", "year": 2022, "value": 10}
            ],
            "CPI_offset": [
                {"year": 2019, "value": -0.001}
            ],
            "EITC_c": [{"EIC": "0kids", "year": 2019, "value": 1000.0}],
            "BEN_ssi_repeal": [
                {"year": 2019, "data_source": "CPS", "value": True}
            ]
        }

        res = helpers.convert_adj(adj, 2019)

>       assert res == {
            "STD": {
                2019: [0, 1, 12268.8, 18403.2, 24537.6],
                2022: [0, 10, 13081.03, 19621.54, 26162.06]
            },
            "CPI_offset": {
                2019: -0.001
            },
            "EITC_c": {
                2019: [1000.0, 3529.87, 5829.75, 6558.98]
            },
            "BEN_ssi_repeal": {
                2019: True
            }
        }
E       AssertionError: assert defaultdict(<...{2019: True}}) == {'BEN_ssi_rep...4, 26162.06]}}
E         Omitting 3 identical items, use -vv to show
E         Differing items:
E         {'EITC_c': {2019: [1000.0, 3538.53, 5844.04, 6575.05]}} != {'EITC_c': {2019: [1000.0, 3529.87, 5829.75, 6558.98]}}
E         Full diff:
E         - defaultdict(<class 'dict'>,
E         + {
E         -             {'BEN_ssi_repeal': {2019: True},...
E         
E         ...Full output truncated (46 lines hidden), use '-vv' to show

cs-config/cs_config/tests/test_functions.py:97: AssertionError
====================================================================================== warnings summary ======================================================================================
/home/hankdoupe/miniconda3/envs/taxbrain-dev/lib/python3.7/site-packages/numba/types/containers.py:3
  /home/hankdoupe/miniconda3/envs/taxbrain-dev/lib/python3.7/site-packages/numba/types/containers.py:3: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working
    from collections import Iterable

/home/hankdoupe/miniconda3/envs/taxbrain-dev/lib/python3.7/site-packages/html5lib/_trie/_base.py:3
  /home/hankdoupe/miniconda3/envs/taxbrain-dev/lib/python3.7/site-packages/html5lib/_trie/_base.py:3: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working
    from collections import Mapping

cs_config/tests/test_functions.py::TestFunctions1::test_run_model
cs_config/tests/test_functions.py::TestFunctions1::test_run_model
cs_config/tests/test_functions.py::TestFunctions1::test_run_model
cs_config/tests/test_functions.py::TestFunctions1::test_run_model
cs_config/tests/test_functions.py::TestFunctions1::test_run_model
cs_config/tests/test_functions.py::TestFunctions1::test_run_model
cs_config/tests/test_functions.py::TestFunctions2::test_run_model
cs_config/tests/test_functions.py::TestFunctions2::test_run_model
cs_config/tests/test_functions.py::TestFunctions2::test_run_model
cs_config/tests/test_functions.py::TestFunctions2::test_run_model
cs_config/tests/test_functions.py::TestFunctions2::test_run_model
cs_config/tests/test_functions.py::TestFunctions2::test_run_model
  BokehDeprecationWarning: 'legend' keyword is deprecated, use explicit 'legend_label', 'legend_field', or 'legend_group' keywords instead

cs_config/tests/test_functions.py::TestFunctions1::test_run_model
cs_config/tests/test_functions.py::TestFunctions1::test_run_model
cs_config/tests/test_functions.py::TestFunctions2::test_run_model
cs_config/tests/test_functions.py::TestFunctions2::test_run_model
  BokehDeprecationWarning: CoffeeScript support is deprecated and will be removed in an eventual 2.0 release. Use JavaScript or TypeScript directly instead.

-- Docs: https://docs.pytest.org/en/latest/warnings.html
=================================================================== 1 failed, 11 passed, 18 warnings in 376.76s (0:06:16) ====================================================================
(taxbrain-dev) hankdoupe@system76-pc:~/Documents/Tax-Brain$ 

After looking at the diff between TC 2.6.0 and 2.7.0, I think the cause is the change to the indexing offset logic (PSLmodels/Tax-Calculator#2381). I'm looking into this now.

hdoupe commented 4 years ago

Can confirm that the difference can be explained by the bug fixed in PSLmodels/Tax-Calculator#2381:

I'll open a PR with the updated values.