PSLmodels / OG-UK

An overlapping generations model to simulate fiscal policy the United Kingdom.
https://pslmodels.github.io/OG-UK
Other
6 stars 7 forks source link

Bump OpenFisca-UK version #39

Closed nikhilwoodruff closed 2 years ago

nikhilwoodruff commented 2 years ago

This PR should resolve issues around working with UK microdata: previously, whenever we made a breaking change to a dataset, everyone working from a previous dataset file found it no longer worked. But now, we've got a new way of handling things that I've tried to make as automatic as possible - bumping openfisca-uk should ensure that the correct dataset re-downloads from GCP or GH releases where necessary.

@rickecon @jdebacker @jpycroft

jdebacker commented 2 years ago

Other than the test failure with the SynthFRS.save() method, this looks good.

nikhilwoodruff commented 2 years ago

Also adding the version check here. @jdebacker I think this will fail this PR but work on future ones, because it compares any changes since the last published tag, but the repo doesn't have any tags yet. This action I've added will publish a new tag for every PR merge commit with the version name.

jdebacker commented 2 years ago

@nikhilwoodruff Thanks for this PR!

I'm seeing test failures on GH Actions and am getting them locally as well. I changed all references to 2018 to 2019 in test_get_micro_data.py as it seemed that was raising errors in that no 2018 file seemed available to download. Doing so fixed one error, but this remains:

(oguk-dev) jason.debacker@JDEBACKER-2 oguk % pytest tests/test_get_micro_data.py
/Users/jason.debacker/anaconda3/envs/oguk-dev/lib/python3.7/site-packages/pep8.py:110: FutureWarning: Possible nested set at position 1
  EXTRANEOUS_WHITESPACE_REGEX = re.compile(r'[[({] | []}),;:]')
=========================================================== test session starts ============================================================
platform darwin -- Python 3.7.10, pytest-5.4.3, py-1.11.0, pluggy-0.13.1
rootdir: /Users/jason.debacker/repos/OG-UK
plugins: xdist-2.5.0, anyio-3.5.0, forked-1.4.0, pep8-1.0.6
collected 3 items

tests/test_get_micro_data.py F..                                                                                                     [100%]

================================================================= FAILURES =================================================================
_________________________________________________________________ test_frs _________________________________________________________________

    def test_frs():
        """
        Check that setting `data` to 'frs' uses cps data
        """
        baseline = False
        start_year = 2019

        # create a parametric reform
        def lower_pa(parameters):
            parameters.tax.income_tax.allowances.personal_allowance.amount.update(
                period=2019, value=10000
            )
            return parameters

        class lower_personal_tax_allowance(Reform):
            def apply(self):
                self.modify_parameters(modifier_function=lower_pa)

        reform = lower_personal_tax_allowance

        calc_out = get_micro_data.get_calculator_output(
>           baseline, start_year, reform=reform, data="frs"
        )

tests/test_get_micro_data.py:47:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
get_micro_data.py:87: in get_calculator_output
    sim = Microsimulation(reform, **sim_kwargs)
../../../anaconda3/envs/oguk-dev/lib/python3.7/site-packages/openfisca_uk/tools/simulation.py:55: in __init__
    f"\nYear {year} not available in dataset {dataset.name}: \n\t* Download the dataset [y]\n\t* Use the synthetic FRS (and set default) [n]\n\nChoice: "
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <_pytest.capture.DontReadFromInput object at 0x7faca85b9e10>, args = ()

    def read(self, *args):
        raise IOError(
>           "pytest: reading from stdin while output is captured!  Consider using `-s`."
        )
E       OSError: pytest: reading from stdin while output is captured!  Consider using `-s`.

../../../anaconda3/envs/oguk-dev/lib/python3.7/site-packages/_pytest/capture.py:733: OSError
----------------------------------------------------------- Captured stdout call -----------------------------------------------------------

Year 2018 not available in dataset synth_frs:
    * Download the dataset [y]
    * Use the synthetic FRS (and set default) [n]

Choice:

========================================================= short test summary info ==========================================================
FAILED tests/test_get_micro_data.py::test_frs - OSError: pytest: reading from stdin while output is captured!  Consider using `-s`.
================================================= 1 failed, 2 passed, 6 warnings in 9.89s ==================================================
(oguk-dev) jason.debacker@JDEBACKER-2 oguk % pytest tests/test_run_example.py

I'm unsure what the traceback in telling me since the start year is 2019, so I don't know where the warning about 2018 not being available is coming from. What do you think?

codecov[bot] commented 2 years ago

Codecov Report

Merging #39 (cabba58) into main (75c844c) will decrease coverage by 0.34%. The diff coverage is 83.33%.

Impacted file tree graph

@@            Coverage Diff             @@
##             main      #39      +/-   ##
==========================================
- Coverage   63.12%   62.77%   -0.35%     
==========================================
  Files           4        4              
  Lines         141      137       -4     
==========================================
- Hits           89       86       -3     
+ Misses         52       51       -1     
Flag Coverage Δ
unittests 62.77% <83.33%> (-0.35%) :arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
setup.py 0.00% <0.00%> (ø)
oguk/get_micro_data.py 70.68% <100.00%> (-0.28%) :arrow_down:
oguk/tests/test_get_micro_data.py 87.50% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update f55e59e...cabba58. Read the comment docs.

nikhilwoodruff commented 2 years ago

Thanks @jdebacker - just took a look. I've changed all 2018 references to 2019 like you, and the GH action now seems to pass without errors - I wonder if my changes are the same as yours?

jdebacker commented 2 years ago

This is great @nikhilwoodruff! Those changes look like mine. Tests are passing here, so I'll merge now. Thanks!