Open andersonfrailey opened 4 years ago
@andersonfrailey, thanks a lot for raising an issue about memory usage in Behavioral-Responses.
Right, we are making the copies to ensure this promise: neither calc_1 nor calc_2 are affected by this response function.
Specifically, the deep copies ensure that the response function doesn't call calc_all()
on calc_1 and calc_2 and leave them in a calculated state when the user isn't expected it. (I suppose they also ensure against some other unintended side effects being added in the future).
So your suggestion makes sense in that it would allow users to explicitly choose whether to allow Behavioral-Responses to calculate their calc_1 and calc_2 or not. They get to ask themselves, do I want a permanent small memory effect or a temporary memory explosion?
On the other hand, adding this option will definitely add some complexity to the code and the user experience. Is it better to just accept and document that a side effect of behavior is calculating your calculators, and then get rid of those two copies for everyone? (I agree with your assessment that we should keep the third).
Interested in other perspectives here.
Also happy to review a PR demonstrating that giving the user the option doesn't add much complexity.
@MattHJensen, thanks for the response. I'll work on a PR that would make the copies optional to see how that affects complexity. For backwards compatibility purposes I'll make it so that the copies are still made by default and the user has to specify they don't want the copies made.
What do you all think about using a keyword argument, inplace
, to indicate that the calculations should be performed on the calculators that are passed to response
, instead of copies? Pandas and NumPy frequently use this keyword for the same purpose. So, it should be familiar to users. Here's an implementation for how this would work: https://github.com/PSLmodels/Behavioral-Responses/compare/master...hdoupe:inplace-flag
I benchmarked the aggregate memory usage using memory_profiler
(version 0.55.0):
inplace |
peak | increment |
---|---|---|
False |
7.056 GB | 6.88 GB |
True |
5.826 GB | 5.652 GB |
using this function:
def test_func(inplace):
rec = tc.Records.cps_constructor()
refyear = 2020
assert refyear >= 2018
reform = {'II_em': {refyear: 1500}}
# ... construct pre-reform calculator
pol = tc.Policy()
calc1 = tc.Calculator(records=rec, policy=pol)
calc1.advance_to_year(refyear)
# ... construct two post-reform calculators
pol.implement_reform(reform)
calc2d = tc.Calculator(records=rec, policy=pol) # for default behavior
calc2d.advance_to_year(refyear)
# Keep track of some of the variables that will be modifed
# in response if inplace is True.
df_before_1 = calc1.dataframe(tc.DIST_VARIABLES)
df_before_2 = calc2d.dataframe(tc.DIST_VARIABLES)
behresp.response(calc1, calc2d, elasticities={"inc": -0.2}, dump=True, inplace=inplace)
# Grab the same variables after.
df_after_1 = calc1.dataframe(tc.DIST_VARIABLES)
df_after_2 = calc2d.dataframe(tc.DIST_VARIABLES)
# If inplace is True, the variables should have been modified.
# If inplace is False, response only modified a copy of calc_1
# and calc_2.
if inplace:
assert not df_before_1.equals(df_after_1)
assert not df_before_2.equals(df_after_2)
else:
assert df_before_1.equals(df_after_1)
assert df_before_2.equals(df_after_2)
which I called like this:
# obtain high-level stats on memory usage
%memit test_func(inplace=False)
At the bottom of this comment, you can find a line-by-line breakdown to see how memory changed in each line of test_func
. You can see that lines 102 and 103 increase the memory usage by about 1.38 GB when inplace
is False
. I'm curious to hear what people think about incorporating inplace
into the API. Even saving 1 GB makes a difference to projects that run Tax-Calculator in parallel like Tax-Brain. For example, if Tax-Brain wants to run 5 years at once, this increases the memory requirement by 5GB.
You can replicate these results with this notebook. Note that the numbers will vary some with each run but the general memory usage stats should remain similar.
With inplace=False
:
Filename: /home/hankdoupe/Documents/Behavioral-Responses/behresp/behavior.py
Line # Mem usage Increment Line Contents
================================================
13 2028.9 MiB 2028.9 MiB def response(calc_1, calc_2, elasticities, dump=False, inplace=False):
14 """
15 Implements TaxBrain "Partial Equilibrium Simulation" dynamic analysis
16 returning results as a tuple of Pandas DataFrame objects (df1, df2) where:
17 df1 is extracted from a baseline-policy calc_1 copy, and
18 df2 is extracted from a reform-policy calc_2 copy that incorporates the
19 behavioral responses given by the nature of the baseline-to-reform
20 change in policy and elasticities in the specified behavior dictionary.
21
22 Note: this function internally modifies a copy of calc_2 records to account
23 for behavioral responses that arise from the policy reform that involves
24 moving from calc1 policy to calc2 policy. Neither calc_1 nor calc_2 need
25 to have had calc_all() executed before calling the response function.
26 By default, neither calc_1 nor calc_2 are affected by this response
27 function. To perform in-place calculations that affect calc_1 and calc_2,
28 set inplace equal to True.
29
30 The elasticities argument is a dictionary containing the assumed response
31 elasticities. Omitting an elasticity key:value pair in the dictionary
32 implies the omitted elasticity is assumed to be zero. Here is the full
33 dictionary content and each elasticity's internal name:
34
35 be_sub = elasticities['sub']
36 Substitution elasticity of taxable income.
37 Defined as proportional change in taxable income divided by
38 proportional change in marginal net-of-tax rate (1-MTR) on taxpayer
39 earnings caused by the reform. Must be zero or positive.
40
41 be_inc = elasticities['inc']
42 Income elasticity of taxable income.
43 Defined as dollar change in taxable income divided by dollar change
44 in after-tax income caused by the reform. Must be zero or negative.
45
46 be_cg = elasticities['cg']
47 Semi-elasticity of long-term capital gains.
48 Defined as change in logarithm of long-term capital gains divided by
49 change in marginal tax rate (MTR) on long-term capital gains caused by
50 the reform. Must be zero or negative.
51 Read response function documentation (see below) for discussion of
52 appropriate values.
53
54 The optional dump argument controls the number of variables included
55 in the two returned DataFrame objects. When dump=False (its default
56 value), the variables in the two returned DataFrame objects include
57 just the variables in the Tax-Calculator DIST_VARIABLES list, which
58 is sufficient for constructing the standard Tax-Calculator tables.
59 When dump=True, the variables in the two returned DataFrame objects
60 include all the Tax-Calculator input and calculated output variables,
61 which is the same output as produced by the Tax-Calculator tc --dump
62 option except for one difference: the tc --dump option provides two
63 calculated variables, mtr_inctax and mtr_paytax, that are replaced
64 in the dump output of this response function by mtr_combined, which
65 is the sum of mtr_inctax and mtr_paytax.
66
67 Note: the use here of a dollar-change income elasticity (rather than
68 a proportional-change elasticity) is consistent with Feldstein and
69 Feenberg, "The Taxation of Two Earner Families", NBER Working Paper
70 No. 5155 (June 1995). A proportional-change elasticity was used by
71 Gruber and Saez, "The elasticity of taxable income: evidence and
72 implications", Journal of Public Economics 84:1-32 (2002) [see
73 equation 2 on page 10].
74
75 Note: the nature of the capital-gains elasticity used here is similar
76 to that used in Joint Committee on Taxation, "New Evidence on the
77 Tax Elasticity of Capital Gains: A Joint Working Paper of the Staff
78 of the Joint Committee on Taxation and the Congressional Budget
79 Office", (JCX-56-12), June 2012. In particular, the elasticity
80 use here is equivalent to the term inside the square brackets on
81 the right-hand side of equation (4) on page 11 --- not the epsilon
82 variable on the left-hand side of equation (4), which is equal to
83 the elasticity used here times the weighted average marginal tax
84 rate on long-term capital gains. So, the JCT-CBO estimate of
85 -0.792 for the epsilon elasticity (see JCT-CBO, Table 5) translates
86 into a much larger absolute value for the be_cg semi-elasticity
87 used by Tax-Calculator.
88 To calculate the elasticity from a semi-elasticity, we multiply by
89 MTRs from TC and weight by shares of taxable gains. To avoid those
90 with zero MTRs, we restrict this to the top 40% of tax units by AGI.
91 Using this function, a semi-elasticity of -3.45 corresponds to a tax
92 rate elasticity of -0.792.
93
94 """
95 # pylint: disable=too-many-locals,too-many-statements,too-many-branches
96
97 # Check function argument types and elasticity values
98 2028.9 MiB 0.0 MiB if inplace:
99 calc1 = calc_1
100 calc2 = calc_2
101 else:
102 2715.8 MiB 686.9 MiB calc1 = copy.deepcopy(calc_1)
103 3409.6 MiB 693.8 MiB calc2 = copy.deepcopy(calc_2)
104 3409.6 MiB 0.0 MiB assert isinstance(calc1, tc.Calculator)
105 3409.6 MiB 0.0 MiB assert isinstance(calc2, tc.Calculator)
106 3409.6 MiB 0.0 MiB assert isinstance(elasticities, dict)
107 3409.6 MiB 0.0 MiB be_sub = elasticities['sub'] if 'sub' in elasticities else 0.0
108 3409.6 MiB 0.0 MiB be_inc = elasticities['inc'] if 'inc' in elasticities else 0.0
109 3409.6 MiB 0.0 MiB be_cg = elasticities['cg'] if 'cg' in elasticities else 0.0
110 3409.6 MiB 0.0 MiB assert be_sub >= 0.0
111 3409.6 MiB 0.0 MiB assert be_inc <= 0.0
112 3409.6 MiB 0.0 MiB assert be_cg <= 0.0
113
114 # Begin nested functions used only in this response function
115 6039.7 MiB 0.0 MiB def _update_ordinary_income(taxinc_change, calc):
116 """
117 Implement total taxable income change induced by behavioral response.
118 """
119 # compute AGI minus itemized deductions, agi_m_ided
120 6039.7 MiB 0.0 MiB agi = calc.array('c00100')
121 6039.7 MiB 0.0 MiB ided = np.where(calc.array('c04470') < calc.array('standard'),
122 6039.7 MiB 0.0 MiB 0., calc.array('c04470'))
123 6039.7 MiB 0.0 MiB agi_m_ided = agi - ided
124 # assume behv response only for filing units with positive agi_m_ided
125 6039.7 MiB 0.0 MiB pos = np.array(agi_m_ided > 0., dtype=bool)
126 6039.7 MiB 0.0 MiB delta_income = np.where(pos, taxinc_change, 0.)
127 # allocate delta_income into three parts
128 # pylint: disable=unsupported-assignment-operation
129 6039.7 MiB 0.0 MiB winc = calc.array('e00200')
130 6039.7 MiB 0.0 MiB delta_winc = np.zeros_like(agi)
131 6039.7 MiB 0.0 MiB delta_winc[pos] = delta_income[pos] * winc[pos] / agi_m_ided[pos]
132 6039.7 MiB 0.0 MiB oinc = agi - winc
133 6039.7 MiB 0.0 MiB delta_oinc = np.zeros_like(agi)
134 6039.7 MiB 0.0 MiB delta_oinc[pos] = delta_income[pos] * oinc[pos] / agi_m_ided[pos]
135 6039.7 MiB 0.0 MiB delta_ided = np.zeros_like(agi)
136 6039.7 MiB 0.0 MiB delta_ided[pos] = delta_income[pos] * ided[pos] / agi_m_ided[pos]
137 # confirm that the three parts are consistent with delta_income
138 6039.7 MiB 0.0 MiB assert np.allclose(delta_income, delta_winc + delta_oinc - delta_ided)
139 # add the three parts to different records variables embedded in calc
140 6039.7 MiB 0.0 MiB calc.incarray('e00200', delta_winc)
141 6039.7 MiB 0.0 MiB calc.incarray('e00200p', delta_winc)
142 6039.7 MiB 0.0 MiB calc.incarray('e00300', delta_oinc)
143 6039.7 MiB 0.0 MiB calc.incarray('e19200', delta_ided)
144 6039.7 MiB 0.0 MiB return calc
145
146 6039.7 MiB 0.0 MiB def _update_cap_gain_income(cap_gain_change, calc):
147 """
148 Implement capital gain change induced by behavioral responses.
149 """
150 6039.7 MiB 0.0 MiB calc.incarray('p23250', cap_gain_change)
151 6039.7 MiB 0.0 MiB return calc
152
153 3618.8 MiB 0.0 MiB def _mtr12(calc__1, calc__2, mtr_of='e00200p', tax_type='combined'):
154 """
155 Computes marginal tax rates for Calculator objects calc__1 and calc__2
156 for specified mtr_of income type and specified tax_type.
157 """
158 3618.8 MiB 0.0 MiB assert tax_type in ('combined', 'iitax')
159 4914.5 MiB 1295.7 MiB _, iitax1, combined1 = calc__1.mtr(mtr_of, wrt_full_compensation=True)
160 5565.7 MiB 651.2 MiB _, iitax2, combined2 = calc__2.mtr(mtr_of, wrt_full_compensation=True)
161 5565.7 MiB 0.0 MiB if tax_type == 'combined':
162 5565.7 MiB 0.0 MiB return (combined1, combined2)
163 return (iitax1, iitax2)
164 # End nested functions used only in this response function
165
166 # Begin main logic of response function
167 3559.4 MiB 149.8 MiB calc1.calc_all()
168 3618.8 MiB 59.4 MiB calc2.calc_all()
169 3618.8 MiB 0.0 MiB assert calc1.array_len == calc2.array_len
170 3618.8 MiB 0.0 MiB assert calc1.current_year == calc2.current_year
171 3618.8 MiB 0.0 MiB mtr_cap = 0.99
172 3618.8 MiB 0.0 MiB if dump:
173 3618.8 MiB 0.0 MiB recs_vinfo = tc.Records(data=None) # contains records VARINFO only
174 3618.8 MiB 0.0 MiB dvars = list(recs_vinfo.USABLE_READ_VARS | recs_vinfo.CALCULATED_VARS)
175 # Calculate sum of substitution and income effects
176 3618.8 MiB 0.0 MiB if be_sub == 0.0 and be_inc == 0.0:
177 zero_sub_and_inc = True
178 if dump:
179 wage_mtr1 = np.zeros(calc1.array_len)
180 wage_mtr2 = np.zeros(calc2.array_len)
181 else:
182 3618.8 MiB 0.0 MiB zero_sub_and_inc = False
183 # calculate marginal combined tax rates on taxpayer wages+salary
184 # (e00200p is taxpayer's wages+salary)
185 3618.8 MiB 0.0 MiB wage_mtr1, wage_mtr2 = _mtr12(calc1, calc2,
186 3618.8 MiB 0.0 MiB mtr_of='e00200p',
187 5565.7 MiB 0.0 MiB tax_type='combined')
188 # calculate magnitude of substitution effect
189 5565.7 MiB 0.0 MiB if be_sub == 0.0:
190 5565.7 MiB 0.0 MiB sub = np.zeros(calc1.array_len)
191 else:
192 # proportional change in marginal net-of-tax rates on earnings
193 mtr1 = np.where(wage_mtr1 > mtr_cap, mtr_cap, wage_mtr1)
194 mtr2 = np.where(wage_mtr2 > mtr_cap, mtr_cap, wage_mtr2)
195 pch = ((1. - mtr2) / (1. - mtr1)) - 1.
196 # Note: c04800 is filing unit's taxable income
197 sub = be_sub * pch * calc1.array('c04800')
198 # calculate magnitude of income effect
199 5565.7 MiB 0.0 MiB if be_inc == 0.0:
200 inc = np.zeros(calc1.array_len)
201 else:
202 # dollar change in after-tax income
203 # Note: combined is f.unit's income+payroll tax liability
204 5565.7 MiB 0.0 MiB dch = calc1.array('combined') - calc2.array('combined')
205 5565.7 MiB 0.0 MiB inc = be_inc * dch
206 # calculate sum of substitution and income effects
207 5565.7 MiB 0.0 MiB si_chg = sub + inc
208 # Calculate long-term capital-gains effect
209 5565.7 MiB 0.0 MiB if be_cg == 0.0:
210 5565.7 MiB 0.0 MiB ltcg_chg = np.zeros(calc1.array_len)
211 else:
212 # calculate marginal tax rates on long-term capital gains
213 # p23250 is filing units' long-term capital gains
214 ltcg_mtr1, ltcg_mtr2 = _mtr12(calc1, calc2,
215 mtr_of='p23250',
216 tax_type='iitax')
217 rch = ltcg_mtr2 - ltcg_mtr1
218 exp_term = np.exp(be_cg * rch)
219 new_ltcg = calc1.array('p23250') * exp_term
220 ltcg_chg = new_ltcg - calc1.array('p23250')
221 # Extract dataframe from calc1
222 5565.7 MiB 0.0 MiB if dump:
223 6283.2 MiB 717.4 MiB df1 = calc1.dataframe(dvars)
224 6279.9 MiB 0.0 MiB df1.drop('mtr_inctax', axis='columns', inplace=True)
225 6276.4 MiB 0.0 MiB df1.drop('mtr_paytax', axis='columns', inplace=True)
226 6276.4 MiB 0.0 MiB df1['mtr_combined'] = wage_mtr1 * 100
227 else:
228 df1 = calc1.dataframe(tc.DIST_VARIABLES)
229 6276.4 MiB 0.0 MiB if not inplace:
230 6276.4 MiB 0.0 MiB del calc1
231 # Add behavioral-response changes to income sources
232 6276.4 MiB 0.0 MiB if inplace:
233 calc2_behv = calc2
234 else:
235 6276.4 MiB 0.0 MiB calc2_behv = copy.deepcopy(calc2)
236 6039.7 MiB 0.0 MiB del calc2
237 6039.7 MiB 0.0 MiB if not zero_sub_and_inc:
238 6039.7 MiB 0.0 MiB calc2_behv = _update_ordinary_income(si_chg, calc2_behv)
239 6039.7 MiB 0.0 MiB calc2_behv = _update_cap_gain_income(ltcg_chg, calc2_behv)
240 # Recalculate post-reform taxes incorporating behavioral responses
241 6039.7 MiB 0.0 MiB calc2_behv.calc_all()
242 # Extract dataframe from calc2_behv
243 6039.7 MiB 0.0 MiB if dump:
244 6756.9 MiB 717.2 MiB df2 = calc2_behv.dataframe(dvars)
245 6753.6 MiB 0.0 MiB df2.drop('mtr_inctax', axis='columns', inplace=True)
246 6750.1 MiB 0.0 MiB df2.drop('mtr_paytax', axis='columns', inplace=True)
247 6750.1 MiB 0.0 MiB df2['mtr_combined'] = wage_mtr2 * 100
248 else:
249 df2 = calc2_behv.dataframe(tc.DIST_VARIABLES)
250 6750.1 MiB 0.0 MiB if not inplace:
251 6750.1 MiB 0.0 MiB del calc2_behv
252 # Return the two dataframes
253 6750.1 MiB 0.0 MiB return (df1, df2)
With inplace=True
:
Filename: /home/hankdoupe/Documents/Behavioral-Responses/behresp/behavior.py
Line # Mem usage Increment Line Contents
================================================
13 2032.4 MiB 2032.4 MiB def response(calc_1, calc_2, elasticities, dump=False, inplace=False):
14 """
15 Implements TaxBrain "Partial Equilibrium Simulation" dynamic analysis
16 returning results as a tuple of Pandas DataFrame objects (df1, df2) where:
17 df1 is extracted from a baseline-policy calc_1 copy, and
18 df2 is extracted from a reform-policy calc_2 copy that incorporates the
19 behavioral responses given by the nature of the baseline-to-reform
20 change in policy and elasticities in the specified behavior dictionary.
21
22 Note: this function internally modifies a copy of calc_2 records to account
23 for behavioral responses that arise from the policy reform that involves
24 moving from calc1 policy to calc2 policy. Neither calc_1 nor calc_2 need
25 to have had calc_all() executed before calling the response function.
26 By default, neither calc_1 nor calc_2 are affected by this response
27 function. To perform in-place calculations that affect calc_1 and calc_2,
28 set inplace equal to True.
29
30 The elasticities argument is a dictionary containing the assumed response
31 elasticities. Omitting an elasticity key:value pair in the dictionary
32 implies the omitted elasticity is assumed to be zero. Here is the full
33 dictionary content and each elasticity's internal name:
34
35 be_sub = elasticities['sub']
36 Substitution elasticity of taxable income.
37 Defined as proportional change in taxable income divided by
38 proportional change in marginal net-of-tax rate (1-MTR) on taxpayer
39 earnings caused by the reform. Must be zero or positive.
40
41 be_inc = elasticities['inc']
42 Income elasticity of taxable income.
43 Defined as dollar change in taxable income divided by dollar change
44 in after-tax income caused by the reform. Must be zero or negative.
45
46 be_cg = elasticities['cg']
47 Semi-elasticity of long-term capital gains.
48 Defined as change in logarithm of long-term capital gains divided by
49 change in marginal tax rate (MTR) on long-term capital gains caused by
50 the reform. Must be zero or negative.
51 Read response function documentation (see below) for discussion of
52 appropriate values.
53
54 The optional dump argument controls the number of variables included
55 in the two returned DataFrame objects. When dump=False (its default
56 value), the variables in the two returned DataFrame objects include
57 just the variables in the Tax-Calculator DIST_VARIABLES list, which
58 is sufficient for constructing the standard Tax-Calculator tables.
59 When dump=True, the variables in the two returned DataFrame objects
60 include all the Tax-Calculator input and calculated output variables,
61 which is the same output as produced by the Tax-Calculator tc --dump
62 option except for one difference: the tc --dump option provides two
63 calculated variables, mtr_inctax and mtr_paytax, that are replaced
64 in the dump output of this response function by mtr_combined, which
65 is the sum of mtr_inctax and mtr_paytax.
66
67 Note: the use here of a dollar-change income elasticity (rather than
68 a proportional-change elasticity) is consistent with Feldstein and
69 Feenberg, "The Taxation of Two Earner Families", NBER Working Paper
70 No. 5155 (June 1995). A proportional-change elasticity was used by
71 Gruber and Saez, "The elasticity of taxable income: evidence and
72 implications", Journal of Public Economics 84:1-32 (2002) [see
73 equation 2 on page 10].
74
75 Note: the nature of the capital-gains elasticity used here is similar
76 to that used in Joint Committee on Taxation, "New Evidence on the
77 Tax Elasticity of Capital Gains: A Joint Working Paper of the Staff
78 of the Joint Committee on Taxation and the Congressional Budget
79 Office", (JCX-56-12), June 2012. In particular, the elasticity
80 use here is equivalent to the term inside the square brackets on
81 the right-hand side of equation (4) on page 11 --- not the epsilon
82 variable on the left-hand side of equation (4), which is equal to
83 the elasticity used here times the weighted average marginal tax
84 rate on long-term capital gains. So, the JCT-CBO estimate of
85 -0.792 for the epsilon elasticity (see JCT-CBO, Table 5) translates
86 into a much larger absolute value for the be_cg semi-elasticity
87 used by Tax-Calculator.
88 To calculate the elasticity from a semi-elasticity, we multiply by
89 MTRs from TC and weight by shares of taxable gains. To avoid those
90 with zero MTRs, we restrict this to the top 40% of tax units by AGI.
91 Using this function, a semi-elasticity of -3.45 corresponds to a tax
92 rate elasticity of -0.792.
93
94 """
95 # pylint: disable=too-many-locals,too-many-statements,too-many-branches
96
97 # Check function argument types and elasticity values
98 2032.4 MiB 0.0 MiB if inplace:
99 2032.4 MiB 0.0 MiB calc1 = calc_1
100 2032.4 MiB 0.0 MiB calc2 = calc_2
101 else:
102 calc1 = copy.deepcopy(calc_1)
103 calc2 = copy.deepcopy(calc_2)
104 2032.4 MiB 0.0 MiB assert isinstance(calc1, tc.Calculator)
105 2032.4 MiB 0.0 MiB assert isinstance(calc2, tc.Calculator)
106 2032.4 MiB 0.0 MiB assert isinstance(elasticities, dict)
107 2032.4 MiB 0.0 MiB be_sub = elasticities['sub'] if 'sub' in elasticities else 0.0
108 2032.4 MiB 0.0 MiB be_inc = elasticities['inc'] if 'inc' in elasticities else 0.0
109 2032.4 MiB 0.0 MiB be_cg = elasticities['cg'] if 'cg' in elasticities else 0.0
110 2032.4 MiB 0.0 MiB assert be_sub >= 0.0
111 2032.4 MiB 0.0 MiB assert be_inc <= 0.0
112 2032.4 MiB 0.0 MiB assert be_cg <= 0.0
113
114 # Begin nested functions used only in this response function
115 4910.2 MiB 0.0 MiB def _update_ordinary_income(taxinc_change, calc):
116 """
117 Implement total taxable income change induced by behavioral response.
118 """
119 # compute AGI minus itemized deductions, agi_m_ided
120 4910.2 MiB 0.0 MiB agi = calc.array('c00100')
121 4910.2 MiB 0.0 MiB ided = np.where(calc.array('c04470') < calc.array('standard'),
122 4910.2 MiB 0.0 MiB 0., calc.array('c04470'))
123 4910.2 MiB 0.0 MiB agi_m_ided = agi - ided
124 # assume behv response only for filing units with positive agi_m_ided
125 4910.2 MiB 0.0 MiB pos = np.array(agi_m_ided > 0., dtype=bool)
126 4910.2 MiB 0.0 MiB delta_income = np.where(pos, taxinc_change, 0.)
127 # allocate delta_income into three parts
128 # pylint: disable=unsupported-assignment-operation
129 4910.2 MiB 0.0 MiB winc = calc.array('e00200')
130 4910.2 MiB 0.0 MiB delta_winc = np.zeros_like(agi)
131 4910.2 MiB 0.0 MiB delta_winc[pos] = delta_income[pos] * winc[pos] / agi_m_ided[pos]
132 4910.2 MiB 0.0 MiB oinc = agi - winc
133 4910.2 MiB 0.0 MiB delta_oinc = np.zeros_like(agi)
134 4910.2 MiB 0.0 MiB delta_oinc[pos] = delta_income[pos] * oinc[pos] / agi_m_ided[pos]
135 4910.2 MiB 0.0 MiB delta_ided = np.zeros_like(agi)
136 4910.2 MiB 0.0 MiB delta_ided[pos] = delta_income[pos] * ided[pos] / agi_m_ided[pos]
137 # confirm that the three parts are consistent with delta_income
138 4910.2 MiB 0.0 MiB assert np.allclose(delta_income, delta_winc + delta_oinc - delta_ided)
139 # add the three parts to different records variables embedded in calc
140 4910.2 MiB 0.0 MiB calc.incarray('e00200', delta_winc)
141 4910.2 MiB 0.0 MiB calc.incarray('e00200p', delta_winc)
142 4910.2 MiB 0.0 MiB calc.incarray('e00300', delta_oinc)
143 4910.2 MiB 0.0 MiB calc.incarray('e19200', delta_ided)
144 4910.2 MiB 0.0 MiB return calc
145
146 4910.2 MiB 0.0 MiB def _update_cap_gain_income(cap_gain_change, calc):
147 """
148 Implement capital gain change induced by behavioral responses.
149 """
150 4910.2 MiB 0.0 MiB calc.incarray('p23250', cap_gain_change)
151 4910.2 MiB 0.0 MiB return calc
152
153 2229.9 MiB 0.0 MiB def _mtr12(calc__1, calc__2, mtr_of='e00200p', tax_type='combined'):
154 """
155 Computes marginal tax rates for Calculator objects calc__1 and calc__2
156 for specified mtr_of income type and specified tax_type.
157 """
158 2229.9 MiB 0.0 MiB assert tax_type in ('combined', 'iitax')
159 3527.8 MiB 1297.8 MiB _, iitax1, combined1 = calc__1.mtr(mtr_of, wrt_full_compensation=True)
160 4199.6 MiB 671.8 MiB _, iitax2, combined2 = calc__2.mtr(mtr_of, wrt_full_compensation=True)
161 4199.6 MiB 0.0 MiB if tax_type == 'combined':
162 4199.6 MiB 0.0 MiB return (combined1, combined2)
163 return (iitax1, iitax2)
164 # End nested functions used only in this response function
165
166 # Begin main logic of response function
167 2170.5 MiB 138.0 MiB calc1.calc_all()
168 2229.9 MiB 59.5 MiB calc2.calc_all()
169 2229.9 MiB 0.0 MiB assert calc1.array_len == calc2.array_len
170 2229.9 MiB 0.0 MiB assert calc1.current_year == calc2.current_year
171 2229.9 MiB 0.0 MiB mtr_cap = 0.99
172 2229.9 MiB 0.0 MiB if dump:
173 2229.9 MiB 0.0 MiB recs_vinfo = tc.Records(data=None) # contains records VARINFO only
174 2229.9 MiB 0.0 MiB dvars = list(recs_vinfo.USABLE_READ_VARS | recs_vinfo.CALCULATED_VARS)
175 # Calculate sum of substitution and income effects
176 2229.9 MiB 0.0 MiB if be_sub == 0.0 and be_inc == 0.0:
177 zero_sub_and_inc = True
178 if dump:
179 wage_mtr1 = np.zeros(calc1.array_len)
180 wage_mtr2 = np.zeros(calc2.array_len)
181 else:
182 2229.9 MiB 0.0 MiB zero_sub_and_inc = False
183 # calculate marginal combined tax rates on taxpayer wages+salary
184 # (e00200p is taxpayer's wages+salary)
185 2229.9 MiB 0.0 MiB wage_mtr1, wage_mtr2 = _mtr12(calc1, calc2,
186 2229.9 MiB 0.0 MiB mtr_of='e00200p',
187 4199.6 MiB 0.0 MiB tax_type='combined')
188 # calculate magnitude of substitution effect
189 4199.6 MiB 0.0 MiB if be_sub == 0.0:
190 4199.6 MiB 0.0 MiB sub = np.zeros(calc1.array_len)
191 else:
192 # proportional change in marginal net-of-tax rates on earnings
193 mtr1 = np.where(wage_mtr1 > mtr_cap, mtr_cap, wage_mtr1)
194 mtr2 = np.where(wage_mtr2 > mtr_cap, mtr_cap, wage_mtr2)
195 pch = ((1. - mtr2) / (1. - mtr1)) - 1.
196 # Note: c04800 is filing unit's taxable income
197 sub = be_sub * pch * calc1.array('c04800')
198 # calculate magnitude of income effect
199 4199.6 MiB 0.0 MiB if be_inc == 0.0:
200 inc = np.zeros(calc1.array_len)
201 else:
202 # dollar change in after-tax income
203 # Note: combined is f.unit's income+payroll tax liability
204 4199.6 MiB 0.0 MiB dch = calc1.array('combined') - calc2.array('combined')
205 4199.6 MiB 0.0 MiB inc = be_inc * dch
206 # calculate sum of substitution and income effects
207 4199.6 MiB 0.0 MiB si_chg = sub + inc
208 # Calculate long-term capital-gains effect
209 4199.6 MiB 0.0 MiB if be_cg == 0.0:
210 4199.6 MiB 0.0 MiB ltcg_chg = np.zeros(calc1.array_len)
211 else:
212 # calculate marginal tax rates on long-term capital gains
213 # p23250 is filing units' long-term capital gains
214 ltcg_mtr1, ltcg_mtr2 = _mtr12(calc1, calc2,
215 mtr_of='p23250',
216 tax_type='iitax')
217 rch = ltcg_mtr2 - ltcg_mtr1
218 exp_term = np.exp(be_cg * rch)
219 new_ltcg = calc1.array('p23250') * exp_term
220 ltcg_chg = new_ltcg - calc1.array('p23250')
221 # Extract dataframe from calc1
222 4199.6 MiB 0.0 MiB if dump:
223 4917.0 MiB 717.4 MiB df1 = calc1.dataframe(dvars)
224 4913.7 MiB 0.0 MiB df1.drop('mtr_inctax', axis='columns', inplace=True)
225 4910.2 MiB 0.0 MiB df1.drop('mtr_paytax', axis='columns', inplace=True)
226 4910.2 MiB 0.0 MiB df1['mtr_combined'] = wage_mtr1 * 100
227 else:
228 df1 = calc1.dataframe(tc.DIST_VARIABLES)
229 4910.2 MiB 0.0 MiB if not inplace:
230 del calc1
231 # Add behavioral-response changes to income sources
232 4910.2 MiB 0.0 MiB if inplace:
233 4910.2 MiB 0.0 MiB calc2_behv = calc2
234 else:
235 calc2_behv = copy.deepcopy(calc2)
236 del calc2
237 4910.2 MiB 0.0 MiB if not zero_sub_and_inc:
238 4910.2 MiB 0.0 MiB calc2_behv = _update_ordinary_income(si_chg, calc2_behv)
239 4910.2 MiB 0.0 MiB calc2_behv = _update_cap_gain_income(ltcg_chg, calc2_behv)
240 # Recalculate post-reform taxes incorporating behavioral responses
241 4910.2 MiB 0.0 MiB calc2_behv.calc_all()
242 # Extract dataframe from calc2_behv
243 4910.2 MiB 0.0 MiB if dump:
244 5627.4 MiB 717.2 MiB df2 = calc2_behv.dataframe(dvars)
245 5624.1 MiB 0.0 MiB df2.drop('mtr_inctax', axis='columns', inplace=True)
246 5620.6 MiB 0.0 MiB df2.drop('mtr_paytax', axis='columns', inplace=True)
247 5620.6 MiB 0.0 MiB df2['mtr_combined'] = wage_mtr2 * 100
248 else:
249 df2 = calc2_behv.dataframe(tc.DIST_VARIABLES)
250 5620.6 MiB 0.0 MiB if not inplace:
251 del calc2_behv
252 # Return the two dataframes
253 5620.6 MiB 0.0 MiB return (df1, df2)
@MattHJensen I'm trying to run Tax-Brain with behresp on my 16GB RAM limited laptop and it's reminding me of this issue. I still think it would be helpful to be able to run behresp without making so many copies of Calculator
objects.
Currently, Behavioral-Responses makes three copies of the calculator objects passed as arguments to the
response
function. The first two are made right when the function starts running, and the third occurs further along when adding the behavioral-response changes to income.It's clear why we would want to make a copy before changing income, and I can see an argument for why we would want to make copies of the original copies from the beginning. However, could we add an argument to the responses function that makes creating these copies optional? From what I can see, there are no modifications being made to the
calc1
andcalc2
variables, only the copy ofcalc2
. Of course I could be missing something obvious.Making the copies optional would significantly reduce the amount of memory needed for partial equilibrium simulations, which would be especially helpful in Tax-Brain.
cc @hdoupe