Possible bug in ConsAggShockModel.py

[edmundcrawley]

Line 542 calculates the KtoY ratio as KtoYnow = np.mean(aAll*pAll), and then the KtoL ratio as: KtoLnow = self.convertKtoY(KtoYnow) This KtoL ratio is then used to calculate interest rates, wages and it is an input into the consumption function.

But in order to be used like this it needs to be normalized by the mean level of permanent income.

I think a simple fix is to change line 542 to:

KtoYnow = np.mean(aAll*pAll)/np.mean(pAll)

Which works for my purposes but when I make this change it seems to break convergence in solving the aggregate market ctswMPC.py.

I will try and figure out why ctswMPC.py stops working, but in the meantime if anyone disagrees/agrees with the above let me know.

[mnwhite]

Yes, there are some problems in the current implementation of AggShockModel, and this is one of them. I have a new/improved/fixed version sitting on my local machine, but haven't got around to issuing a PR for it; I hope to do that this weekend (there are a lot of other changes as well, so it will take some time to move everything from my personal HARK to the main repo).

The math in the current public version is wrong due to an error in my reasoning / transcribing from Mathematica code.

Convergence problems when normalizing by mean permanent income likely arise due to noisiness in np.mean(pAll). In theory, this average is always 1.0 exactly, but the finite sim population mean it's not. Extra jumpiness in the history of KtoL means that the autoregression on KtoL jumps around more and possibly never converges.

[edmundcrawley]

Thanks Matt, it looks like that is likely the issue with convergence. Although won't no.mean(pAll) vary with the aggregate permanent shocks even if we have an infinite number of agents?

I'm getting stuck into the AggShockModel right now. It would be useful to know what the known problems are so I can bear them in mind and/or provide a fix myself.

One other thing I noticed was that when agents die they are replaced with an agent who has permanent income of 1. In my code I've changed that to have them receive the mean permanent income of the period they are born. Can I suggest that as a change to be incorporated?

[mnwhite]

Sorry, yes, mean permanent income should move aggregate permanent shocks. I hope to have my proposed fixes issued as a pull request this weekend. I'll try to prioritize it.

"Newborns" being assigned the current mean permanent income is obviously feasible. My first thought is that it isn't particularly important, as the aggregate shocks are small so mean pLvl is never very far from 1.0, and proportionally few agents die each period. You could use parameters where that's not the case, but they wouldn't be particularly realistic.

The second thing here is that the big PR coming this weekend includes an overhaul / unification of simulation across models, moving to more OOP. To use the "universal simulator", a subclass of AgentType should specify methods called simDeath and simBirth which describe (respectively) how agents are selected to be replaced and how new agents are created. In consumption saving models, simBirth needs to know the distribution of assets and the distribution of permanent income for newborns; these are attributes called aNrmInitMean, etc. Adding the feature that permanent income of newborns should be adjusted according to the mean level of survivors is just a tack-on two lines to simBirth.

I don't want to force that adjustment automatically, because we're moving towards the aggregate shock model being able to handle lifecycle consumers rather than just "perpetual youth" consumers. Newborns shouldn't get mean permanent income in that model, obviously.

On Thu, Oct 13, 2016 at 3:00 PM, edmundcrawley notifications@github.com wrote:

Thanks Matt, it looks like that is likely the issue with convergence. Although won't no.mean(pAll) vary with the aggregate permanent shocks even if we have an infinite number of agents?

I'm getting stuck into the AggShockModel right now. It would be useful to know what the known problems are so I can bear them in mind and/or provide a fix myself.

One other thing I noticed was that when agents die they are replaced with an agent who has permanent income of 1. In my code I've changed that to have them receive the mean permanent income of the period they are born. Can I suggest that as a change to be incorporated?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/econ-ark/HARK/issues/88#issuecomment-253606706, or mute the thread https://github.com/notifications/unsubscribe-auth/ANUQFeQB8_jD8W4EQHrU21APzGgREckIks5qzn--gaJpZM4KWNLD .

[edmundcrawley]

The shocks are small, but if there is growth in the economy then over time the mean permanent income grows much larger than 1. In my example I have 1% growth, which over a simulation of 3000 quarters leads to permanent incomes of the newborns being far far below the mean. Furthermore, in this setup aggregate growth in steady state is zero.

Even in a lifecycle model we are likely to want to be able to set a steady growth of income from one generation to the next, so something equivalent will be desired.

I look forward to your release!

[mnwhite]

Yes, we do want to allow for growth between cohorts, but this can be achieved by using cohort scaling factors: when aggregating, an agent that is n periods older than the youngest agent should be interpreted as having a permanent income level G^{-n} times its pLvl, because it was born a while ago and doesn't get as large a cohort bonus. Your method is also valid in this case, but not necessarily in the models where permanent income is an explicit state variable in the decision problem, rather than normalized out (and only tracked in the background); the state grid for pLvl needs to be the same across all cohorts, so "cohort income growth" has to be applied later.

I'm not sure that made any sense.

On Thu, Oct 13, 2016 at 3:43 PM, edmundcrawley notifications@github.com wrote:

The shocks are small, but if there is growth in the economy then over time the mean permanent income grows much larger than 1. In my example I have 1% growth, which over a simulation of 3000 quarters leads to permanent incomes of the newborns being far far below the mean. Furthermore, in this setup aggregate growth in steady state is zero.

Even in a lifecycle model we are likely to want to be able to set a steady growth of income from one generation to the next, so something equivalent will be desired.

I look forward to your release!

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/econ-ark/HARK/issues/88#issuecomment-253617817, or mute the thread https://github.com/notifications/unsubscribe-auth/ANUQFTYoWKPQ2aX07bf9zYs1x7o_W6LVks5qzonUgaJpZM4KWNLD .

[edmundcrawley]

Got it - sounds like you are addressing the issues I am having. Thanks.

[edmundcrawley]

I don't think the simulation overhaul has fixed this bug. KtoLNow needs to account for the permanent aggregate shocks to effective labor supply. This can be fixed by removing line 622 AggregateL = 1 and replacing it after PermShkAggNow is defined (say on line 628) with AggregateL = np.mean(pLvlNow)*PermShkAggNow To do this you need to add pLvlNow back into reap_vars.

pLvlNow can be very different from 1, so this leads to serious miscalculations of wages and interest rates in simulations.

Strictly, AggregateL should account for the mortality process so: AggregateL = LivProb_np.mean(pLvlNow)_PermShkAggNow + (1-LivProb)*meanPermIncomeofNewBorns but the effect here is small.

[mnwhite]

Edmund's PR that changes the aggregate state variable from KtoL to Magg has been merged. Closing issue.