search

PolicyEngine / policyengine-us

The PolicyEngine US Python package contains a rules engine of the US tax-benefit system, and microdata generation for microsimulation analysis.
https://policyengine.org/us
GNU Affero General Public License v3.0
95 stars 167 forks source link

Fix TAXSIM validation tests on MacOS #1415

Open rickecon opened 1 year ago

rickecon commented 1 year ago

Two questions:

  1. Why does test_against_taxsim.py skip the tests for the Windows CI (see line 34)?
  2. Even though we cannot do CI testing on Mac OS X, is there any reason why this test should not work running locally on Mac OS X (see description below)?

My fork of openfisca-us is up-to-date with the main branch as of today. When I run make test, everything passes except that I get five failures in test_against_taxsim.py.

openfisca_us/tests/test_variables.py ....................................................................... [  6%]
............................................................................................................ [ 16%]
............................................................................................................ [ 27%]
............................................................................................................ [ 37%]
............................................................................................................ [ 47%]
............................................................................................................ [ 57%]
............................................................................................................ [ 67%]
............................................................................................................ [ 78%]
............................................................................................................ [ 88%]
............................................................................................................ [ 98%]
..                                                                                                           [ 98%]
openfisca_us/tests/code_health/parameters.py .                                                               [ 98%]
openfisca_us/tests/code_health/variable_names.py .                                                           [ 98%]
openfisca_us/tests/microsimulation/test_against_taxsim.py FFFFF                                              [ 99%]
openfisca_us/tests/microsimulation/test_microsim.py .                                                        [ 99%]
openfisca_us/tests/microsimulation/data/test_imports.py ..                                                   [ 99%]
openfisca_us/tests/microsimulation/data/acs/test_acs.py ss                                                   [ 99%]
openfisca_us/tests/microsimulation/data/cps/test_cps.py ..                                                   [100%]

I checked the most recent PRs to openfisca-us, and the test_against_taxsim.py tests pass on the Linux CI tests (and are skipped for the Windows CI tests). Is this an issue with running these TAXSIM tests on Mac OS X?

cc: @nikhilwoodruff

nikhilwoodruff commented 1 year ago

Hey Rick,

I get the same results. The core reason is because the package has to download a different executable of TAXSIM depending on the OS, and I hadn't rigorously tested MacOS and Windows prior to now because they're not essential and we can run the tests on Linux. But I'm now on MacOS so I'll take a look and see what's going on.

rickecon commented 1 year ago

Thanks, @nikhilwoodruff. That explains it.