IRA accounts with early withdrawal penalties

[pmgblz]

We would like to simulate IRA accounts with early withdrawal penalties. Our simulation would require the following three extensions to HARK. Do you have already coded up any of the following extensions (but not submitted to HARK) or could share advice on how to create these? Any help would be greatly appreciated!

Here are the extensions:

• [ ] Model two savings accounts.

• [ ] Each savings account is kinked; i.e. potentially a different borrowing rate than interest rate on saving.

• [ ] Ideally, a penalty for withdrawing before a certain date or age. An alternative would be to create time varying interest rates for borrowing and savings.

[mnwhite]

These are not actively being worked on at this time, nor do I think I've made an issue specifically pertaining to this. We do want to add models that have multiple assets, rather than the single asset models that are currently in HARK. These generally come in two flavors:

1) Portfolio allocation problems between a risky and riskless asset. There is an issue for this, and I had a grad student working on it this summer/fall. However, she did not complete the work.

2) Retirement-style accounts, like you propose here.

The HARK team has discussed (2) being a to-do item, with my preference being to implement the example model from Jorgensen and Druedahl's G2EGM paper (JEDC 2017). IIRC, their example model has an inaccessible retirement account, but substantially the same solution method is used if you institute an early withdrawal penalty instead.

I have coded up versions of the G2EGM algorithm (for other models) in OpenCL, interfaced with HARK via the opencl4py package. The "upper envelope" step of G2EGM can be a bit cumbersome, so it's much nicer to have this piece done in a lower level language. That code is hidden in my private fork of HARK (for personal research projects), but I'm planning to write a generic version of it and move it into the main repo sometime in the spring.

However, I don't anticipate that we would merge that branch into the master branch until we get around to "package-ifying" HARK and making it able to be installed by pip. We have tried to make HARK use only packages that come with Anaconda automatically; currently the only exception is joblib for multithreading. Making HARK an install-able package is a priority item for early 2018.

My advice to you is to very carefully read this paper (http://www.sciencedirect.com/science/article/pii/S0165188916301920) and understand the G2EGM solution method. I anticipate that this would be the most efficient way to solve the model you describe, given the multiple kink features. And from a modeling perspective: the IRA account should not have a kink but a bound at zero. You can't borrow on a retirement account with the promise of putting the money back later.

[nomadj1s]

Hello,

I'm on the research team that originally made this request. Thank you for the references. They were very helpful.

Instead of using the G2EGM model, which leverages a functional form assumption to get a closed form solution, I am looking at the Nested Endogenous Grid Method (NEGM) here, also by Jeppe Druedahl. This method relaxes the functional form assumption, uses EGM to pin down the consumption function, but then uses value function iteration (VFI) to pin down the other endogenous variable, in our case deposits or withdrawals from the illiquid account.

My question: are there already tools within the HARK tool kit to implement VFI, or do I need to code that up on my own? If not, do you have a reference for the frontier of VFI?

Thanks

[mnwhite]

Hey there! Patrick Mogensen, a PhD student of Jeppe, is currently working on porting a general-use G2EGM utility tool into HARK. I believe that work is approaching completion.

Once G2EGM is done, he will turn to making a general-purpose NEGM utility, which (if I understand correctly) will incorporate a value-maximization component.* The horizon for that is still 2-3 months away by my estimation.

I don't personally use value maximization (with respect to continuous controls) in my own work, so I don't know what the frontier of methods is. Patrick has been flagged to respond to this thread, and he will probably know more. With the proper incantations, Jeppe could be summoned as well.

- I don't call this step VFI, as that implies (to me) a backward iteration process across many periods. The explicit value function step in NEGM maximizes control-conditional value within* a period.

[nomadj1s]

Great thanks for the response. I will await to see if Patrick chimes in. I also agree that the maximization step in NEGM may be a bit different than standard VFI.

[pkofod]

Instead of using the G2EGM model, which leverages a functional form assumption to get a closed form solution,

It's a bit more than that really, but I understand why the different things might be a bit hard to separate based on reading the paper alone. G2EGM's main point is basically to divide potential problematic cases into "segments" according to different margins being constrained (constrained post decision cash-on hand, constrained retirement savings, everything unconstrained, ...). With a solution for each segment, you then do a simultaneous upper envelope and re-interpolation down to a common grid. It is true that there are closed form expression for each segment, but I guess some of that could still be numerically solved for, and you would still label the solution method as G2EGM, as long as you used the re-interpolate/envelope step (which is what I see as "the G2EGM" method, the authors might have a more nuanced view, I'm actually unsure).

I am looking at the Nested Endogenous Grid Method (NEGM) here, also by Jeppe Druedahl.

Good, I'm sure Jeppe would appreciate knowing that people are reading he manuscript, he's certainly quite enthusiastic about NEGM himself.

My question: are there already tools within the HARK tool kit to implement VFI, or do I need to code that up on my own? If not, do you have a reference for the frontier of VFI?

No, there isn't. And I think that MNW's following way of characterizing is the correct answer to your question.

*- I don't call this step VFI, as that implies (to me) a backward iteration process across many periods. The explicit value function step in NEGM maximizes control-conditional value within a period.

I think it still remains to be seen when and where which method might shine. It's a bit hard to say much without any details about your models. What functional form assumptions are you uncomfortable with in their paper? I don't recognize neither your names nor your picture, so I can't deduce much from that, but if you want to share something in private, that's fine as well. Fewer people will be able to chime in though.

[nomadj1s]

Thanks for the input. To clarify, here are the two features of G2EGM that I'm alluding to, as discussed in the Nested Endogenous Grid Method (NEGM) paper Jeppe Druedahl:

1. G2EGM requires that the equation system of stacked discrepancies in the first order conditions and post-decision state equations have a unique solution. (pg. 4-5)

2. G2EGM rel[ies] on a complex multi-dimensional upper envelope algorithm, while NEGM rel[ies] on a simple one-dimensional upper envelope. (pg. 5)

The first is achieved in the specific application of the G2EGM paper by assuming the return to assets in the pension account has a strictly concave functional form, i.e. the g(d) function. This delivers a closed form solution for optimal deposits/withdrawals for that account, i.e. equation 3.7 in the G2EGM paper. The specific functional form is not as important as having g'(d) be a function of d.

Both NEGM and G2EGM involve an upper envelope step, but the one in the NEGM approach is not as complicated/doesn't involve constructing triangles or barycentric interpolation. The tradeoff is that NEGM requires a value function maximization step.

The model I'm working on is a standard consumption-savings model, with a liquid account that has interest factor Rboro for borrowing and Rsave for saving, and an illiquid account (IRA) with interest factor Rira. The illiquid account has a minimum balance of zero, and there is a proportional penalty, t, for making withdrawals from the illiquid account before a T_ira. We also impose Rboro > Rira > Rsave.

[pkofod]

doesn't involve constructing triangles or barycentric interpolation. The tradeoff is that NEGM requires a value function maximization step

This part is real... I'm still polishing off the implementation I've been working on, but it's really hard to get some weird behavior here, so I'd be very interested in seeing a horse race between the two (I might be the one doing it, so :) )

[nomadj1s]

Hello,

I have finished a first draft of this model on a branch I created. I have a few follow up questions:

1. Because this model uses the NEGM method, it doesn't exactly nest the standard ConsIndShockModel.py, but in theory, if I shut down the illiquid asset, I should get very similar results to the ConsIndShockModel.py. My question: what is the best way to diagnose my model? Are there particular functions or figures that I should create using my model and the ConsIndShockModel.py to see if I am at least getting the basics right?

2. Once I think the model is up and working, is there a way to share it and have people test it out/suggest improvements?

Thanks

[mnwhite]

1. Being 100% sure that your model is really doing what it's "supposed" to do, and that you didn't make some strange error or overlook some issue, is impossible. It's the major existential crisis that you've signed up for by pursuing this line of research.

As you suggest, the first sanity check is to see if your model reproduces the results (policy function, etc) of some simpler, known model when certain parameters are trivialized to shut down the new features. A basic two asset model with early withdrawal penalties on one (the "retirement fund") turns into ConsIndShock when the withdrawal penalty is set to zero (or the "retirement age" when the withdrawal penalty is removed anyway is set at 0 or 1) and the interest rate on the retirement fund equals Rfree in ConsIndShock (and the interest rate on the other asset is lower). It is never optimal to put any money in the lower return asset, and agents will save in the retirement fund as they would in ConsIndShock.

Beyond that, how to "diagnose" that a model is working correctly is a weird blend of art and science. You can play with parameters and make sure that the model results move in the "right" direction according to your intuition. Or you can try to calculate some object in the model using multiple approaches (a "clever" and "dumb" way) and see if they align. What to do is model-specific.

---

2. If your fork is public, then this is easy: Just tell the world about it, and they can simply merge a branch from your fork into a branch on their fork of HARK and test it out. If your fork is private, then you would have to issue a PR into master HARK. Then it's possible to merge your branch in locally and people who follow econ-ark/HARK to play with your branch.

[pkofod]

Beyond that, how to "diagnose" that a model is working correctly is a weird blend of art and science. You can play with parameters and make sure that the model results move in the "right" direction according to your intuition. Or you can try to calculate some object in the model using multiple approaches (a "clever" and "dumb" way) and see if they align. What to do is model-specific.

As you suggest, the first sanity check is to see if your model reproduces the results (policy function, etc) of some simpler, known model when certain parameters are trivialized to shut down the new features.

This is Keane's "make it flat"-advise.

Beyond that, I'd say post it if possible, and we (I? :) ) can have a look.

[nomadj1s]

Thanks for the feedback. I agree with your suggestions. I will make my branch public here, but before doing so, I'd like to run the simplified version and see how well it matches a ConsIndShock model. This will take a day or so.

[pkofod]

Thanks for the feedback. I agree with your suggestions. I will make my branch public here, but before doing so, I'd like to run the simplified version and see how well it matches a ConsIndShock model. This will take a day or so.

Great. While I've had some issues with G2EGM (most are sorted out / understood now), it could potentially still be interesting to match them up against eachother.

[nomadj1s]

Hi All,

I'm now ready to share the code I've been working on. I'm still relatively new to GitHub, but I'll try my best to give you all the info you need to check out the model.

1. Repository: I've forked the HARK repo and it lives here:

   https://github.com/nomadj1s/HARK

2. In order to get started with HARK, I created a branch called "demo." Due to a lack of foresight, I never changed the name, so this branch doesn't have a particularly useful name.

3. In the ConsumptionSaving folder, you will see a filed named ConsIRAModel.py. This file implements a consumption-savings model with two types of savings account. One is a liquid savings account that is kinked: the interest on borrowing is higher than the interest on saving. The other account acts like an IRA: it's balance cannot go below zero, it has a higher interest rate than the liquid account, there is an early withdrawal penalty where a fraction t of the withdrawal is forfeited, the early withdrawal penalty is in effect up until a specific age, and there is a maximum amount that can be deposited into the account each period. The model uses the Nested Endogenous Grid Method (NEGM) to solve for the optimal behavior of an agent.

4. Notation:

   a. There are two control variables: c and d, which stand for consumption and deposits/withdrawals from the IRA account, respectively. Both variables are normalized, relative to the permanent income level in a given period.

   b. There are two main state variables: m and n, which are the amount of liquid assets (potentially negative) and the amount of illiquid assets (non-negative) at the beginning of the period. These are also normalized. An additional level of assets, l, measure liquid assets on hand, net of any deposit or withdrawal made into the illiquid account. For example, if a deposit of d is made, then l is defined as: l = m - d. When withdrawals are made, the previous equation has to also take into account the penalty for early withdrawals.

   c. The after decision asset levels are denoted a for liquid resources and b for illiquid resources. The model takes as parameters grids for a and b. If the number of grid points for b are set to zero (i.e. the parameter bXtraGrid is an empty array), then the model collapses to a standard consumption-savings problem, and deposits d are constrained to be zero.

   d. The method used to solve the model is the NEGM, presented in Druedahl (2018). The model essentially proceeds in two steps. First, a "pure consumption" function is estimated, which takes as fixed illiquid assets, and solves for optimal consumption. This step uses the endogenous grid method, with some adjustments to handle non-convexities and corner solutions, to choose the pure consumption function. Second, it is noted that conditional on m and d, we are left with the pure consumption problem. So, we then do a grid search over d, and, using the pure consumption function to pin down consumption after d is chosen, find the optimal d and, by extension, optimal c. Many of the steps of the solver try follow as closely as possible the recipe presented in Druedahl (2018).

5. Differences from HARK:

   a. Because there is a grid search involved, the model can take much longer to solve. A 30 period model can take about 36 hours to run. The key step that takes longer is the grid search over d. In order to speed up the model, I use parallel processing, using the pathos package for python. The parameters of the ConsIRAModel allow the user to turn on or off multiprocessing. Using multiprocessing via pathos the code takes about 5 hours to solve a 30 period model (using a computer with essentially 16 cpus).

   b. Also, because the model can take so long, I edited the core.py file to include a tool from the progressbar package. This gives a status on the code, letting the user know how much time is left until the code is finished.

6. Comparison to HARK: As discussed above, one way to check the code is to create a version of the IRA model that should have the same patterns as the HARK model. I did this in two ways:

   a. First, I set the grid for the b variable to empty, which constrains the agent to setting d to zero in every period and not using the illiquid account. Because this case doesn't require the full NEGM, the model essentially uses a standard endogenous grid method (EGM) to solve. The only difference is how corner solutions are handled. Here, I've graphed the consumption functions in period 15, 20, and 25, of a 30 period model. The consumptions functions line up pretty well, except right near the kink:

   

   

   

   Since the IRA model almost completely collapses to EGM in this case, it is perhaps not surprising that they track so well.

   b. Next, I allow the IRA model more ability to stray from the regular Kinked model. Here, I set the IRA interest rate to 0.02, and set the liquid account interest rate on savings to 0.00. I also set the penalty on early withdrawals from the IRA off. In retrospect, I should have also lifted the cap on deposits into the IRA, but didn't think to do it this time. We still get that the consumptions functions generally track each other. However, this time around, the IRA model returns shakier results near the kink in the standard consumption function from the Kinked HARK model:

   

   

   

7. Beyond this comparison, I run the full blown IRA model, with 30 periods. However, I am getting some unexpected results. Here I plot consumption, liquid assets, and illiquid assets over time:

   

   Some things make sense here: for example the agent starts to borrow heavily from the liquid account once the early withdrawal penalty is removed (vertical red line) and finances that with in part by borrowing from the illiquid account. However, it is not clear to me why the agent doesn't liquidate the account in the final period. I also ran the standard HARK Kinked model, to see if all of these models "leave" money on the table in the last period. This is true for the Kinked HARK model. Here is a plot:

   

   Perhaps this is a more general question for the HARK forum, but why don't resources get completely consumed in the last period. Note: I did not know which is the proper "final period," in a model with 30 periods, the simulation seems to have people expire at age 31. Should I count this extra year as the true final period?

8. Please test it out: Right now, I am not getting results that match the real world data as well as I would like. For example, there is a marked increase in withdrawing money from an IRA after the penalty is lifted, when you look at real world tax returns. I'm not sure if this has to do with the parameters used to initiate the consumer type, or issues with the code or model. It would be helpful for people to point out incorrect coding or typos or errors that might prevent the IRA model from being more useful.

Thanks again for the help in getting this model started, and I appreciate any feedback you may have.

[nomadj1s]

Hello @mnwhite and @pkofod. Not sure if either of you saw this post. Of course, this was posted close to the holiday season, and, of course, you all have other things to do, but just flagging in case you didn't see.

[mnwhite]

I saw, was just in the middle of holidays and then AEA travel. I will be putting a lot more time into HARK this semester, so I will give this a proper response in a few weeks.

But on first glance: Nice!

On Wed, Jan 9, 2019 at 5:53 PM nomadj1s notifications@github.com wrote:

Hello @mnwhite https://github.com/mnwhite and @pkofod https://github.com/pkofod. Not sure if either of you saw this post. Of course, this was posted close to the holiday season, and, of course, you all have other things to do, but just flagging in case you didn't see.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/econ-ark/HARK/issues/136#issuecomment-452897994, or mute the thread https://github.com/notifications/unsubscribe-auth/ANUQFcXTp2b4NOTxChRG8zooRv9CXTmgks5vBnEigaJpZM4Q4pPu .

[nomadj1s]

Ok, thanks. No worries, no rush.

[llorracc]

Better than nice, it is super impressive! (Like Matt, I've been busy with holidays, AEA meetings, and other things, but will have much more time this semester).

I'll ask @pkofod to take a look at it too, since it is somewhat related to things he and I have been working on.

PS. The default "terminal" consumption rule is to consume everything, so I'm guessing that your puzzle might just be that what you are seeing is the amount of liquid assets at the beginning rather than at the end of the last period? Otherwise, there is indeed something funny going on...

[pkofod]

PS. The default "terminal" consumption rule is to consume everything, so I'm guessing that your puzzle might just be that what you are seeing is the amount of liquid assets at the beginning rather than at the end of the last period? Otherwise, there is indeed something funny going on...

Hm, it still seems weird in the kinked model, because even if it's a timing issue, the consumption and beginning of period assets don't match up? The assets are larger than the consumption even in period 31. I'll have a look at what's going on here.

[pkofod]

Ok, thanks. No worries, no rush.

Yeah sorry, I did actually look at it, just never got to produce a response, sorry. I'll return with a proper response / review asap.

I must say I am a bit confused with the very long run times! I'll have to take a look at the code to understand why your grid search is so expensive (and necessary).

It's easier to do a code review if you make a PR though. Say I want to make the comment that https://github.com/econ-ark/HARK/compare/master...nomadj1s:demo#diff-67fb27aa1943f1987ffa99891b88c08fR57 is a big no-no, then it's much easier to do it in a PR (btw, the problem there is that you're just assigning to variable a the HARKobject class it self, so you're modifying the HARKobject class. You should create a new class that inherits from HARKobject and add paramters to that.)

[pkofod]

Some things make sense here: for example the agent starts to borrow heavily from the liquid account once the early withdrawal penalty is removed (vertical red line) and finances that with in part by borrowing from the illiquid account. However, it is not clear to me why the agent doesn't liquidate the account in the final period. I also ran the standard HARK Kinked model, to see if all of these models "leave" money on the table in the last period. This is true for the Kinked HARK model. Here is a plot:

Could you provide the exact code that you ran to get that plot? Just so we're looking at the same thing.

[nomadj1s]

Hi @pkofod,

Thanks for taking a look, and sorry the delayed response. In response to both of your posts:

1. Is a PR a pull request? I'm new to github. How would I go about doing a pull request? Since I have cloned the HARK repo and also made a branch, is the pull request asking for the branch to be merged into the main HARK in my cloned repo? Since I've changed some of the main files for HARK (i.e. added a timer display) I'm not sure I'd want to merge the branch back yet, if that makes sense. In any event, can you point me in the right the direction to make a pull request?

2. To make the plot, this is what I did: a. At the bottom of this file, you can see in the main section that I run the IRAConsumerType. There is also commented code that runs the KinkedConsumerType. I then collect the consumption functions, evaluated at a set of asset points, and export a csv IRA_Kinked_data.csv. b. Next, I ran a Stata do file to make the plots. Since I'm new to Python, it was much easier for me to make the graphs in Stata. That do file is here. If you prefer for me to put the plotting code in Python, I can.

[nomadj1s]

@pkofod,

I gave you the code for the wrong plot. The code to make the plots over time is here. At the bottom of the ConsIRAModel.py, you can see that I ran a simulation and exported the results into csv files. Then the do file I've linked to takes that data and makes different plots over time. To make the plot you are referring do, I ran the KinkedConsumerType and did essentially the same thing. I can upload that code as well if you'd like. The part where I made the figures is less cleanly documented, because I switched to working off of GitHub and quickly making some figures for review.

[pkofod]

@nomadj1s I'm working on putting the various discrete/continuous choice things I've been involved with into proper classes these days. I just wanted to let you know that I'm a bit surprised about the 30 hour runtimes you've mentioned. You said it was because of the grid search, but I cannot replicate such long run times even if I also use grid search to do the withdrawal step (when you already have your consumption function).

Consider the following figure. It comes from a model with a liquid and an illiquid asset (savings accounts). What you're looking at is the adjustment policy, that is how much to take out (if negative) our put into (if positive) an illiquid savings account with a fixed adjustment rate relative to permanent income. The plateau with 0 change represent the (liquid, illiquid) points where no adjustment is made to the stock of the illiquid savings.

So this should be relatively comparable to your problem in terms of components that need to be handled. However, 30 periods on my old laptop runs in just minutes, so I think there's something you need to look into in your code. Did you use the Iskhakov, Jørgensen, Rust, Schjerning upper envelope algorithm for the consumption function solve? Or is all this because you're actually doing 2 levels of grid searches?

[nomadj1s]

@pkofod,

Thanks again for taking a look. I have made some updates to the code in response to a number of comments you have made, and I also have some responses for your other questions:

1.

Could you provide the exact code that you ran to get that plot? Just so we're looking at the same thing.

I've rewritten a version of the code, ConsIndShockModelPlot.py, that solves the KinkedRconsumerType and plots a time series of consumption, beginning of the period assets, and end of the period assets. Here is the plot: I can confirm that the consumption function for the last period, period 30, involves consuming all of the assets, but for some reason, the simulation ends in period 11 with positive assets, on average. You can see in that file the code that was used to make the plot. I ran a simulation, and average by age, across the simulated agents.

2. Did you use the Iskhakov, Jørgensen, Rust, Schjerning upper envelope algorithm for the consumption function solve? Or is all this because you're actually doing 2 levels of grid searches?

   I use the Nested Endogenous Grid Method of Druedahl (2018). The first step is an endogenous grid method. I can confirm that the first step is not what is causing the long time to solve. The second step is a grid search over deposit amounts. That is what is holding me up. I sped it up by using parallel processing, but without that, the code takes quite long. This is the line of the code:

   1224    dNrm_list = [[self.findArgMaxv(m,n) for m in mNrm] 
   1225    for n in nNrm]

   The method findArgMaxv returns the optimal deposit d using this line:

   1185    d = basinhopping(self.makeNegvOfdFunc,
   1186    0.0,minimizer_kwargs={"bounds":((-nNrm + 1e-10,
   1187    self.MaxIRA),),
   1118    "args":(mNrm,nNrm)}).x

   In order to make sure I am not finding a local optimum, I use the basinhopping command from scipy.optimize. This solver takes longer than normal, because it chooses random starting points within the grid to avoid missing a global optimum. There are two state variables, m and n, so the optimal d has to be found for each combination of these two assets, the liquid and illiquid, respectively. The makeNegvOfdFunc is negative 1 times the value function given d, n, and m, where c is pinned down conditional on these three variables, using the consumption function solved for in the first step.

3. It's easier to do a code review if you make a PR though.

   How do I make the code a PR? Does PR stand for "pull request?" As I mentioned above, the code is a branch of a cloned repository of HARK. Do I have to make you a collaborator on the cloned repo in order to send you a pull request? Apologies, but I am new to GitHub.

4. Say I want to make the comment that master...nomadj1s:demodiff-67fb27aa1943f1987ffa99891b88c08fR57 is a big no-no, then it's much easier to do it in a PR (btw, the problem there is that you're just assigning to variable a the HARKobject class it self, so you're modifying the HARKobject class. You should create a new class that inherits from HARKobject and add paramters to that.)

   When I click on this link, I am not taken to any particular code, but just page that says "There isn't anything to compare." I don't see anywhere in the code where I am using a variable named a. Can you clarify?

5. One final note: I've also rewritten the code where I compare the IRAConsumerType to the KinkedRconsumerType. It's called ConsIRAModelVsConsIndShockModel.py. It now creates all of the figures from within python.

Thanks again.

[MridulS]

@Mv77 is working on a revised version of the portfolio model that may be able to handle a version of this.

[nomadj1s]

Hello @mnwhite , @pkofod , @Mv77: Sorry to revive this thread after many years, but I am writing back to see if you are aware of any implementations of the NEGM model in HARK, or other models that can estimates two-asset models, one illiquid, and one liquid and possibly kinked?