MattHJensen
commented
6 years ago From @donboyd5
I'll try to pull together my thoughts, notes, and programs on the state-assignment issue. Let me give a few overview thoughts now.
First, I use the term state distribution rather than state assignment to reflect the fact that a return might be distributed to several states, partially (e.g., 20% to this state, 80% to that state), rather than assigned entirely to one state.
There are 2 approaches I can think of: (1) the one I have been using - distribute selected returns to all states, and (2) the one the Urban Institute has been using - distribute all returns to all states. I don't have an a priori opinion on which is better, but their's does create a large file and I like parsimony.
1) My approach - distribute selected returns to all states (let's say 50 states and 70k returns) Two major steps (many substeps): Step a) Distribute PUF un-state-coded returns to the 50 states. a.i) Chose the the proportion of each return to be distributed to each state, such that: a.ii) Each such proportion is in [0, 1] a.iii) The sum of proportions for every return is 1 (i.e., there are 70k adding-up constraints) a.iv) The results hit known/estimated targets for each state for maybe 20 variables (e..g, # of returns with cap gains, total dollar of gains, # of returns with SALT deduction, total SALT deduction) based on published SOI aggregates. These are constraints to be satisfied. There might be 1,000 such constraints (20 constraints per state, 50 states) a.v) The proportions are chosen to optimize some objective function intended to minimize "distortion" from these distributions or that has other nice qualities. (Dan F. likes entropy maximization. I like something else. I have done it both ways. It is worth discussing, but it is a single technical choice within a much larger methodology.)
This is therefore a nonlinear program (NLP). It is rather large. I gave you 2 different dimensions on the phone. The first was correct, the 2nd wasn't. It has about 3.5 million variables (one proportion for each of 70k returns x 50 states) and 71k constraints. It can use a lot of memory but there are software solutions to that problem. It can be hard to solve. But it is solvable.
Step b) Supplement the already-state-coded returns (the other maybe 90k returns - I don't remember how many). Turns out you can't really hit SOI aggregates for the 50 states for some variables based on the already-state-coded returns. What I did is beef up the number of state-coded returns for each state and income range by pulling in copies of similar returns from other states, based on a distance function. Then I adjusted weights to hit known SOI values by state and income range, using an NLP approach similar to that in Step a. There are a lot of important details involved in doing this in a workable way.
2) Urban Institute approach For each state, the reweight the PUF so that the weighted returns hit known targets for each income range, without paying any attention to state codes that are on the file. I think they use an LP or NLP approach to reweighting. Assuming the numbers above (160k PUF returns), this results in 160k returns per state, or 8 million returns - many of which have zero or near-zero weight. Personally, I would weed out near-zero returns and reweight again.
I don't know which approach is better. Mine says, in Step b, that there is useful information in the available state coding and we should use it, supplemented. Theirs says there is no value in state codes, and we don't care how much we use any one return.
I think the two approaches are worth some debate, and some analytic comparisons.
There are many other important issues as you forecast from a known year to the unknown future (e.g., what to do about reported pass-through income in Kanas?).
From me:
Thank you — I am also quite excited. State distribution would satisfy a large fraction of our user requests that currently go unsatisfied.
We will be working with PUFs from 2009 on, none of which have state codes for any records. Is it right to think that the Urban Institute approach will be our only option?
From Don:
My initial reaction is yes, as long as you don't want to impose any constraints such as requiring that returns used to represent one state cannot also be used to represent another state. Under these circumstances (no state codes at all), I don't think I see a reason to impose such a constraint.
My own sense of tidiness would make me want to have a smaller file than Urban has, rather than using all returns for each state. You'll probably end up with a lot of records for a given state that represent a tiny fraction of a person. It might be worth some intellectual and experimentation effort to investigate ways to drop a lot of these super-small returns.
All of this is worth discussion.
I think if you don't have any state codes, and you go in this direction, the problem should become quite easy computationally. It might go something like this (let's say there are 150k records in the PUF):
For each state X:
Start with all 150k PUF records
Scale their weights proportionately so that the weights add to the number of returns in the state as reported in SOI summary data (e.g., if California is, say, 1/6 of the number of returns in the U.S., then divide each record weight by 6). Possibly do this for subgroups for which we have SOI summary data on # of returns - for example, for each of 5 income ranges by 2 marital classes (married-joint and other). Thus, the scaled file would have the right number of weighted total returns in each of, say, 10 mutually exclusive categories. (The weighted totals for income, deductions, etc., would be way off.)
Choose 150k record-specific scaling factors, to be multiplied by each record weight, such that when weights are adjusted, we hit known SOI targets for the state. The targets might be items such as, for each of 5 income ranges: num married returns, num other returns (already correct at this stage) total agi, total wages, total cap gains, total itemized deductions, total SALT deduction num returns with wages, num returns with cap gains, num itemizers, num SALT takers and so on - maybe 20 or 30 of these
Call the factors we are choosing x[i] where i runs from 1 to 150k Choose these x[i] such that each x[i] >=0 and each x[i]<= some upper bound, such as 10 (150k bound constraints)
Establish constraints that ensure the targets are hit (within tolerances). For example, if there is $15 billion of agi in the $50-75k agi range in state X, then the constraint looks like:
lefthand side:
sum x[i] w[i] {agi[i] >= 50e3 & agi[i] < 75e3) over i
and the right hand side is
15e9
We choose the x's to satisfy these constraints and bounds, in a way that optimizes some penalty function. This is tricky because we don't really have a priori knowledge of what the x's should be. One penalty function would be that we want the x's to be near 1 -- so that the adjusted weights should be near the weights we created in step 2.
If so, then we might penalize differences from 1. One objective function (to be minimized) would be:
sum (x[i] - 1) ^2 over i
but others could be of interest. (As noted, Dan Feenberg likes entropy maximization.)
There is some art in this step because some targets could be hard to hit.
- After doing steps 1-3, we'd have a reweighted file with 150k records that hits the targets for state X. But a lot of the new weights would be VERY small (e.g., when doing this for Mississippi, the weights for records with high SALT deductions might have very low weights). It might be worth considering reasonable ways to pare down the number of records - possibly dropping a lot of records and repeating steps 1-3. My guess is that you could often get down to much smaller numbers of records for each state. Might be worthwhile.
All of this is worth debate.
From me:
Could you verify that my understanding is correct?
- we will be adding ~51 columns to the database representing the state weights (or, alternatively, they could contain scaling factors to be applied to the standards weights to obtain state weights)
- If we don’t ‘drop' the records for a given state that represent a tiny fraction of a person, then all records will have a non-zero value in each of those ~51 cols.
- If we do ‘drop' the records for a given state that represent a tiny fraction of a person, then some records will have zeroes in one or more of those ~51 cols.
- One advantage of dropping the records is that tabbing federal revenues by state might be somewhat faster to perform.
- A more significant advantage of dropping the records is that state calculators could, potentially, be significantly more efficient as they could ignore the 0 weight rows.
From Don:
we will be adding ~51 columns to the database representing the state weights (or, alternatively, they could contain scaling factors to be applied to the standards weights to obtain state weights)
Yes, if done the way Urban Institute does it. I think of this as a "wide" file.
An alternative is to have a "long" file where there are 51 times as many records -- one set of records for each state, with different weights or adjustment factors for each state. The potential advantage of this is you could then change not just the weights but the actual record values (e.g., agi or the SALT deduction or whatever). Not sure this would ever be needed but worth discussion. For example, what do we do if we are projecting data for Kansas forward from the latest PUF year, and we have their state income tax law that encouraged people to recharacterize what might have been wage income as, instead, pass-through income. IF we had appropriate targets, we might find that the only sensible way to hit them was not just to adjust weights, but even (perhaps) to adjust values so that some records have more pass-through income and less wages. Maybe. I don't know. But at some point, a good topic of discussion.
And more generally, as you move from: a) the first stage, where you are adjusting the PUF to hit known SOI aggregates from the PUF year, to b) the second stage, where you are projecting records to a future year, with no known targets (but with targets you may estimate), you may want the flexibility to adjust not just the weights on records, but also the values. In that case you may no longer have (or want to have) identical records for every state, with different weights; you might want different values, too. At this point, it probably makes sense to move from the wide (51 columns) format to the long format (51 times as many records, each with a state code).
If we don’t ‘drop' the records for a given state that represent a tiny fraction of a person, then all records will have a non-zero value in each of those ~51 cols.
Not necessarily. Depends on optimization choices: (a) bounds on the weight adjustment factors, and (b) the objective function.
Call the weight adjustment factors, x[i, j] where (i in 1:150k, if there are 150k records in the PUF now, and j in 1:51 if there are 51 "states"). Then, if:
a) the variable bounds are set so that x[i, j] in [0, large number] for all i, j, (i.e., a weight adjustment factor is allowed to be zero), and
b) the objective function does not penalize x[i, j]==0 excessively, then some (or even a lot, depending on the objective function) of the x[i, j] could be zero. With Dan Feenberg's objective function, all x[i, j] will be nonzero. (In his function, when distributing returns across states, we are minimizing the sum over i, j of x[i, j]*ln(x[i, j], and since we can't take natural logs of zero, we can't have x[i, j]==0 for any i, j.) But with other objective functions, a lot of the x[i, j]'s could be zero.
If we do ‘drop' the records for a given state that represent a tiny fraction of a person, then some records will have zeroes in one or more of those ~51 cols.
Yes, by definition. One advantage of dropping the records is that tabbing federal revenues by state might be somewhat faster to perform. Yes, although I don't think computation cost will matter much for most kinds of income tax models.
A more significant advantage of dropping the records is that state calculators could, potentially, be significantly more efficient as they could ignore the 0 weight rows.
Yes, but again, computation cost should not be a big deal.
I think the real questions are tidiness and communication:
Tidyness: Does it bother us that we have a lot of records - perhaps tens of thousands - that are tiny fractions of a person and that in aggregate may not even add up to a single person? Do we want to say that this tax overhaul option will result in 0.005 people who are losers, or whatever? I am not sure. For some reason it bothers me, but maybe it shouldn't.
Communication: Even if it does not bother us, will it be hard to explain it to some audiences, and to gain their confidence in the results? If so, we could just not make a big deal of it. Or we could consider whether we could have equally valid results from a more parsimonious sample (i.e., with some zero-weight records, which are then dropped).
From me:
These answers are very helpful.
I am convinced by the value of the “long” file approach.
I am not so ready to write off computation costs as a consideration since some of our users’ applications call Tax-Calculator many times. (see, e.g., this discussion). I’d add efficiency as another benefit in favor of parsimony!
Do you mind if I move this discussion onto GitHub so that others can participate? I would just need a GitHub handle for you.
From Don:
Yes, I am a lover of parsimony and efficiency. I have spent days making computer programs more efficient (I guess we can debate whether spending that much of my time is true efficiency, but I find that efficiency has many benefits, including greater readability, less likelihood of error). And if we find reasons to do stochastic runs - for example, running 51 state models 1,000 times, then efficiency will definitely be computationally important.
Moving to github would be great; my username is donboyd5.
Whew!
donboyd5
ernietedeschi
MaxGhenis
feenberg
One of the most frequent requests that I have heard from users, especially during the TCJA debate, has been for the capability to analyze the impact of federal tax reform by state.
Don Boyd (@donboyd5) and I have been discussing an approach to do this, and he has given me permission to move our conversation onto GitHub. I will reproduce the conversation to date in the next comment.
Another advantage of the approach that we discuss is that it would provide a PUF-based dataset that could support for state-level calculators.