Add capability to handle tmd input

[martinholmer]

Add code to handle new private tmd.csv in a manner like the private puf.csv is handled.

The private tmd.csv file contains 2021 data and has its own public tmd_weights.csv.gz file containing weights for 2021 through 2074. Also, the new public growfactors.csv file has factors extending through 2074.

These three files are created in the tax-microdata repository using the following scripts:

• create_taxcalc_input_variables.py produces the private tmd.csv file
• create_taxcalc_sampling_weights.py produces the public tmd_weights.csv.gz file
• create_taxcalc_growth_factors.py produces the public growfactors.csv file

The code in this pull request still specifies the Policy.LAST_BUDGET_YEAR to be 2034; a subsequent pull request will increase it to 2074 (which has already been tested).

[codecov[bot]]

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 99.42%. Comparing base (c339245) to head (6a869c6). Report is 1 commits behind head on master.

Additional details and impacted files

[](https://app.codecov.io/gh/PSLmodels/Tax-Calculator/pull/2740?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=PSLmodels) ```diff @@ Coverage Diff @@ ## master #2740 +/- ## ======================================= Coverage 99.42% 99.42% ======================================= Files 13 13 Lines 2594 2603 +9 ======================================= + Hits 2579 2588 +9 Misses 15 15 ``` | [Flag](https://app.codecov.io/gh/PSLmodels/Tax-Calculator/pull/2740/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=PSLmodels) | Coverage Δ | | |---|---|---| | [unittests](https://app.codecov.io/gh/PSLmodels/Tax-Calculator/pull/2740/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=PSLmodels) | `99.42% <100.00%> (+<0.01%)` | :arrow_up: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=PSLmodels#carryforward-flags-in-the-pull-request-comment) to find out more. | [Files](https://app.codecov.io/gh/PSLmodels/Tax-Calculator/pull/2740?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=PSLmodels) | Coverage Δ | | |---|---|---| | [taxcalc/data.py](https://app.codecov.io/gh/PSLmodels/Tax-Calculator/pull/2740?src=pr&el=tree&filepath=taxcalc%2Fdata.py&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=PSLmodels#diff-dGF4Y2FsYy9kYXRhLnB5) | `99.24% <ø> (ø)` | | | [taxcalc/growdiff.py](https://app.codecov.io/gh/PSLmodels/Tax-Calculator/pull/2740?src=pr&el=tree&filepath=taxcalc%2Fgrowdiff.py&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=PSLmodels#diff-dGF4Y2FsYy9ncm93ZGlmZi5weQ==) | `100.00% <100.00%> (ø)` | | | [taxcalc/growfactors.py](https://app.codecov.io/gh/PSLmodels/Tax-Calculator/pull/2740?src=pr&el=tree&filepath=taxcalc%2Fgrowfactors.py&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=PSLmodels#diff-dGF4Y2FsYy9ncm93ZmFjdG9ycy5weQ==) | `100.00% <100.00%> (ø)` | | | [taxcalc/records.py](https://app.codecov.io/gh/PSLmodels/Tax-Calculator/pull/2740?src=pr&el=tree&filepath=taxcalc%2Frecords.py&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=PSLmodels#diff-dGF4Y2FsYy9yZWNvcmRzLnB5) | `100.00% <100.00%> (ø)` | | | [taxcalc/taxcalcio.py](https://app.codecov.io/gh/PSLmodels/Tax-Calculator/pull/2740?src=pr&el=tree&filepath=taxcalc%2Ftaxcalcio.py&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=PSLmodels#diff-dGF4Y2FsYy90YXhjYWxjaW8ucHk=) | `100.00% <100.00%> (ø)` | |

[jdebacker]

@martinholmer I've reviewed this PR and had one small question left in my review.

But there is a larger issue that this PR touches on that might be worth discussion before merging. There has been work in the past (see PR #2538) to disentangle the tax calculator logic from the data used for specific calculations. Things like the cps_constructor method and the default use of the puf.csv are a convenience, but come at a cost of making implicit assumptions about the quality of certain data files that are outside the scope of Tax-Calculator. This PR adds additional methods for convenience with a(nother) particular data file. I can see the advantage of these conveniences, but on the other hand it does push in the opposite direction that the project was hoping to go. cc @MattHJensen thoughts?

[martinholmer]

@jdebacker said in PR #2740:

But there is a larger issue that this PR touches on that might be worth discussion before merging. There has been work in the past (see PR https://github.com/PSLmodels/Tax-Calculator/pull/2538) to disentangle the tax calculator logic from the data used for specific calculations. Things like the cps_constructor method and the default use of the puf.csv are a convenience, but come at a cost of making implicit assumptions about the quality of certain data files that are outside the scope of Tax-Calculator. This PR adds additional methods for convenience with a(nother) particular data file. I can see the advantage of these conveniences, but on the other hand it does push in the opposite direction that the project was hoping to go.

There has been no work on PR #2538 in over three years. Seems like there is little or no momentum in the "direction that the project was hoping to go".

[jdebacker]

Thanks of the PR @martinholmer. Merging.