PolicyEngine / policyengine-us

The PolicyEngine US Python package contains a rules engine of the US tax-benefit system, and microdata generation for microsimulation analysis.
https://policyengine.org/us
GNU Affero General Public License v3.0
102 stars 175 forks source link

Simulate TAXSIM35's incorrect PA income tax forgiveness calculations for validation #1518

Closed martinholmer closed 1 year ago

martinholmer commented 1 year ago

Using the 0.177.8 version of PolicyEngineUS and the 10/24/22 version of TAXSIM35, there are differences in the forgiveness amount between the two models. After a detailed examination of the test case provided below, it appears as if PolicyEngineUS is calculating the legally correct forgiveness amount. The difference is caused by TAXSIM35 calculating a continuous forgiveness rate rather than the legally correct stair-step function being calculated by PolicyEngineUS.

This is the first instance of situations that will, no doubt, arise often in the future as the scope of the PolicyEngineUS vs TAXSIM35 validation testing broadens to include more states. This kind of problem was first addressed last July in issue #1090.

Here is the test case and then the PA income tax differences:

- name: Tax unit with taxsimid 13632 in r21.its.csv and r21.ots.csv
  absolute_error_margin: 0.01
  period: 2021
  input:
    people:
      person1:
        is_tax_unit_head: 1
        is_tax_unit_spouse: 0
        is_tax_unit_dependent: 0
        age: 74
        employment_income: 26000
        taxable_interest_income: 5010
        social_security: 24000
        real_estate_taxes: 23000
        interest_expense: 17000
        ssi: 0  # not in TAXSIM35
        state_supplement: 0  # not in TAXSIM35
        wic: 0  # not in TAXSIM35
      person2:
        is_tax_unit_head: 0
        is_tax_unit_spouse: 1
        is_tax_unit_dependent: 0
        age: 74
        employment_income: 22000
        ssi: 0  # not in TAXSIM35
        state_supplement: 0  # not in TAXSIM35
        wic: 0  # not in TAXSIM35
      person3:
        is_tax_unit_head: 0
        is_tax_unit_spouse: 0
        is_tax_unit_dependent: 1
        age: 11
        ssi: 0  # not in TAXSIM35
        state_supplement: 0  # not in TAXSIM35
        wic: 0  # not in TAXSIM35
      person4:
        is_tax_unit_head: 0
        is_tax_unit_spouse: 0
        is_tax_unit_dependent: 1
        age: 16
        ssi: 0  # not in TAXSIM35
        state_supplement: 0  # not in TAXSIM35
        wic: 0  # not in TAXSIM35
      person5:
        is_tax_unit_head: 0
        is_tax_unit_spouse: 0
        is_tax_unit_dependent: 1
        age: 16
        ssi: 0  # not in TAXSIM35
        state_supplement: 0  # not in TAXSIM35
        wic: 0  # not in TAXSIM35
      person6:
        is_tax_unit_head: 0
        is_tax_unit_spouse: 0
        is_tax_unit_dependent: 1
        age: 16
        ssi: 0  # not in TAXSIM35
        state_supplement: 0  # not in TAXSIM35
        wic: 0  # not in TAXSIM35
    spm_units:
      spm_unit:
        members: [person1, person2, person3, person4, person5, person6]
        snap: 0  # not in TAXSIM35
        tanf: 0  # not in TAXSIM35
    tax_units:
      tax_unit:
        members: [person1, person2, person3, person4, person5, person6]
        tax_unit_childcare_expenses: 1000
        premium_tax_credit: 0  # not in TAXSIM35
        pa_use_tax: 0  # not in TAXSIM35
    households:
      household:
        members: [person1, person2, person3, person4, person5, person6]
        state_code: PA
  output:  # expected results from TAXSIM35
    pa_income_tax: 1308.44

The test results are:

================================= FAILURES ===================================
_______________________________ test session _________________________________
/Users/mrh/work/Policy-Engine-US/PA/r21-id13632.yaml:
  Test 'Tax unit with taxsimid 13632 in r21.its.csv and r21.ots.csv':
    pa_income_tax@2021: [1464.6663] differs from 1308.43994140625
                        with an absolute margin [156.22632] > 0.01
========================= short test summary info ============================
FAILED r21-id13632.yaml::
============================ 1 failed in 0.37s ===============================
MaxGhenis commented 1 year ago

Thanks, sounds like this is another instance where a taxsim reform would aid in our validation (#1090). It's also consistent with taxsim avoiding stepped policies in the case of the EITC investment income limit.