Spousal age in the CPS file

[ernietedeschi]

I'm matching raw CPS microdata from 2013-15 to the tc cps.csv file using the h_seq, ffpos, and a_lineno variables. I do a second round of matching off of the a_spouse variable in the CPS microdata to bring the same variables of interest into the tc cps.csv spousal records.

As a validation cross-check, I compared the age of each record in the CPS microdata to that in the tc cps.csv

For tax unit heads, the age match between the CPS microdata and the tc cps.csv is 100%.

For spouses, the match is only 97.8% however.

In looking at the underlying misses, the cps.csv file has some implausible values for spouse age.

For example, the spouse for RECID 292373 has an age of 3. In RECID 294599, the age is 7.

Some other cps.csv spouses have ages of 0 despite being present in the matched CPS record and the unit being correctly coded as MARS = 2 in the cps.csv. RECID 292658 is an example of this.

Most of the misses are plausible value in their own right but still different from the matched CPS record, sometimes significantly.

Are these deviations intentional?

[andersonfrailey]

Thanks for pointing this out, @evtedeschi3. The deviations are not intentional. My intuition says this is probably caused by the scripts misidentifying the spouse when creating the record and assigning the wrong age. I'll look into this more and see if I can find the problem.

[martinholmer]

@andersonfrailey, I've checked the CPS spouse_age problems that @evtedeschi3 first identified in #225 using the newest CPS data. There are still problems. First I show my tabulations (by MARS) of the unzipped cps.csv.gz file from Tax-Calculator release 0.20.2, and then I offer some observations.

iMac:Tax-Calculator mrh$ ./csv_vars.sh cps.csv | grep -e age -e MARS
1 age_head
2 age_spouse
42 MARS

iMac:Tax-Calculator mrh$ awk -F, 'NR>1{t++;n[$42]++}END{for(i in n)print i,n[i];print t}' cps.csv
2 252988
4 22006
1 181471
456465

iMac:Tax-Calculator mrh$ awk -F, 'NR>1&&$42!=2{n[$2]++;t++}END{for(i in n)print i,n[i];print t}' cps.csv
0 203477
203477

iMac:Tax-Calculator mrh$ awk -F, 'NR>1&&$42==2{n[$2]++}END{for(i in n)print i,n[i]}' cps.csv | awk '{printf("%02d\t%d\n",$1,$2)}' | sort | head -20
00  8455
01  150
02  168
03  221
04  183
05  184
06  121
07  162
08  138
09  126
10  173
11  135
12  199
13  113
14  174
15  180
16  150
17  188
18  221
19  312

iMac:Tax-Calculator mrh$ awk -F, 'NR>1&&$42==2{n[$2]++}END{for(i in n)print i,n[i]}' cps.csv | awk '{printf("%02d\t%d\n",$1,$2)}' | sort | tail -20
62  5196
63  4785
64  4482
65  5288
66  4782
67  4427
68  3638
69  3050
70  3202
71  3347
72  3045
73  2351
74  1815
75  1848
76  1704
77  1506
78  1362
79  1304
80  3975
85  2616

So, we can see that spouse_age is zero in all the filing units that are not MARS==2 (married filing jointly), which is as it should be. So, everything is good so far. But when we tabulate the distribution of spouse_age for those with MARS==2, we see sensible counts for older ages, but not so sensible counts for younger ages. In particular, there are 8455 filing units with MARS==2 and spouse_age==0. And then there are more than a few filing units who have an implausibly low values for spouse_age. The lowest spouse_age value in the puf.csv data file for filing units with MARS==2 is 15 years old.

Now that you've successfully completed all the recent enhancements to the taxdata repo, it seem like fixing this CPS spouse_age problem should have a high priority. In particular, I think this CPS spouse_age problem needs to be fixed before we consider moving to a more recent CBO projection (as proposed in #180.

What are you thoughts on taxdata development? Are there any other things that need to be fixed?

[andersonfrailey]

@martinholmer the next biggest step in taxdata development from my perspective is replacing the SAS code to make the CPS file with Python. I put that on hold the last couple of weeks to work on PUF development, but I've reached a point where I have almost everything written and have moved on to squishing bugs that result in major differences between the current CPS and what I get from the Python scripts.

I think it will be easier to solve this problem with spouse age when we have everything running in Python. I'd say it will be at least two or three more weeks before I'm ready to open a pull request though. Would you say this issue is high enough priority to try and fix it in the SAS code?

[ernietedeschi]

I assume the priority here is to get the code in Python first before worrying about integrating the 2016 and 2017 ASEC releases?

I've been correcting the spousal errors by just importing the age recorded in the CPS ASECs for those records using the household and family identifiers. That seems to work fine for the moment.

[martinholmer]

@andersonfrailey said:

I think it will be easier to solve this problem with spouse age [in the cps.csv.gz file] when we have everything running in Python. I'd say it will be at least two or three more weeks before I'm ready to open a pull request though. Would you say this issue is high enough priority to try and fix it in the SAS code?

No. And @evtedeschi3 seems to agree, which is far more important. So, let's wait for the Python CPS-creation code to be active and then solve the spouse_age problem identified in issue #225.