llorracc
commented
3 months ago Seb,
I know you’ve been working on a whole new scheme for simulation. Is this from that new scheme, or from our existing simulation tools?
From: Sebastian Benthall @.> Date: Tuesday, February 6, 2024 at 15:16 To: econ-ark/HARK @.> Cc: Subscribed @.***> Subject: [econ-ark/HARK] PermShk's are not random enough (Issue #1379)
In some circumstances, our consumer AgentTypes are, in simulation, experiencing more highly correlated permanent income shocks than is probable.
This seems to be happening closer to the beginning of a simulation, in the first N periods. It depends somewhat but not entirely on the random seeds chosen.
Here is an example that exposes the issue:
First, create two identical ConsumerTypes that differ only in a random seed. Solve and simulate them.
from HARK.ConsumptionSaving.ConsPortfolioModel import SequentialPortfolioConsumerType
a1 = SequentialPortfolioConsumerType(seed = 1000, T_sim = 30, AgentCount = 5, UnempPrb = 0.0)
a1.solve()
a1.track_vars += ["pLvl", "mNrm", "cNrm", "Share", "Risky", "PermShk"]
a1.initialize_sim()
a1.simulate()
a2 = SequentialPortfolioConsumerType(seed = 200, T_sim = 30, AgentCount = 5, UnempPrb = 0.0)
a2.solve()
a2.track_vars += ["pLvl", "mNrm", "cNrm", "Share", "Risky", "PermShk"]
a2.initialize_sim()
a2.simulate()
a1.history['PermShk'] - a2.history['PermShk']
This last line compares the PermShk permanent incomes shocks. We expect these shocks to be idiosyncratically drawn from a lognormal distribution that is discretized to 7 values.
Here is what this is generating:
array([[ 0. , 0. , 0. , 0. , 0. ],
[ 0. , 0. , 0. , 0. , 0. ],
[ 0. , 0. , 0. , 0. , 0. ],
[ 0. , 0. , 0. , 0. , 0. ],
[ 0. , 0. , 0. , 0. , 0. ],
[ 0. , 0. , 0. , 0. , 0. ],
[ 0. , 0. , 0. , 0. , 0. ],
[ 0. , 0. , 0. , 0. , 0. ],
[ 0. , 0. , 0. , 0. , 0. ],
[ 0. , 0. , 0. , 0. , 0. ],
[ 0. , 0. , 0. , 0. , 0. ],
[ 0. , 0. , 0. , 0. , 0. ],
[ 0. , 0. , 0. , 0. , 0. ],
[ 0. , 0. , 0. , 0. , 0. ],
[ 0. , 0. , 0. , 0. , 0. ],
[ 0.17305358, 0. , 0. , 0. , 0. ],
[-0.08931416, -0.12008051, -0.04086614, 0.07720723, 0. ],
[ 0. , 0. , 0. , 0. , 0. ],
[ 0. , 0. , -0.14608218, 0. , 0. ],
[-0.17305358, 0.17305358, 0.07720723, -0.16094665, 0.12008051],
[ 0.10974109, -0.18380317, -0.1353326 , 0.08931416, 0. ],
[ 0.04601844, 0. , 0.03772098, 0.03634109, 0. ],
[-0.20939467, 0.25026081, 0.06887496, -0.31913576, 0.20939467],
[-0.03634109, 0.03634109, 0.04086614, -0.07720723, -0.08373942],
[ 0.04601844, 0.07406208, 0.20939467, 0.10974109, -0.31913576],
[ 0. , 0. , 0. , 0. , 0. ],
[ 0. , 0. , 0. , 0. , 0. ],
[ 0. , 0.03772098, 0. , 0. , 0. ],
[ 0. , -0.11492821, 0. , 0.11492821, 0.1353326 ],
[-0.31913576, 0.18380317, -0.11492821, 0. , 0.07720723]])Notice that for 19 out of 30 periods, the permanent income shocks are identical between the two simulation runs, across 5 agents. The odds of this happening with equiprobable distributions are (1/7)^5^19, which is an extremely small number.
With different random seeds, the problem is somewhat reduced, but even picking random numbers for the seeds exhibits this phenomenon much more than expected. I have been unable to find seeds that do not get perfect shock matches between the two simulations in the second period.
— Reply to this email directly, view it on GitHubhttps://github.com/econ-ark/HARK/issues/1379, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AAKCK72EWRK55GZIMSL7WRDYSKFTRAVCNFSM6AAAAABC4T5JIKVHI2DSMVQWIX3LMV43ASLTON2WKOZSGEZDCNJZGQ4DCOI. You are receiving this because you are subscribed to this thread.Message ID: @.***>
sbenthall
In some circumstances, our consumer AgentTypes are, in simulation, experiencing more highly correlated permanent income shocks than is probable.
This seems to be happening closer to the beginning of a simulation, in the first N periods. It depends somewhat but not entirely on the random seeds chosen.
Here is an example that exposes the issue:
First, create two identical ConsumerTypes that differ only in a random seed. Solve and simulate them.
This last line compares the
PermShkpermanent incomes shocks. We expect these shocks to be idiosyncratically drawn from a lognormal distribution that is discretized to 7 values.Here is what this is generating:
Notice that for 19 out of 30 periods, the permanent income shocks are identical between the two simulation runs, across 5 agents. The odds of this happening with equiprobable distributions are (1/7)^5^19, which is an extremely small number.
With different random seeds, the problem is somewhat reduced, but even picking random numbers for the seeds exhibits this phenomenon much more than expected. I have been unable to find seeds that do not get perfect shock matches between the two simulations in the second period.