distribute_params and random seeds -- a truly random random seed?

[sbenthall]

The distribute_params method is designed to let you clone an AgentType while distributing the new Agents over a distribution over a single parameter, such as DiscFac.

https://github.com/econ-ark/HARK/blob/910bff34d6e1d59e686029076ca82a6cf81553c2/HARK/core.py#L1664

Using this feature, I've been surprised when the resulting AgentTypes all have perfectly correlated income shocks.

In hindsight, I should not have been surprised: these AgentTypes all have the same random seed.

However, correcting for this issue required some hacky custom code, looping over the agents, changing their seeds, reseting their RNGs, etc.

While having default and steady random seeds is very useful for testing and development, in Monte Carlo simulations with mutliple runs of the same model, you want actually random random seeds.

HARK could better support this with some kind of flag in the AgentType constructor that, if True, gave the agent a random seed (using the larger computing system to provide the entropy).

A corresponding argument could be added to `distribute_params``

[mnwhite]

I'm not sure this is hacky code at all, and it's intended behavior. In each of my applied projects, I do exactly what you describe: after creating all of my types, I loop over them and assign a new unique seed value. There's no need to resetRNG(), as that's done automatically by initializeSim().

It's fine if we build in options to do this automatically, but "truly random seeds" should not be part of the default functionality. With structural estimation, you want "replicable randomness": pseudo-random sequences that will be drawn in the same way no matter when or how you run the code. Some projects have required semi-mind-bending coding to make sure that tiny changes in parameters don't lead to chaotic effects solely through changes in the RNG.

On Thu, Oct 21, 2021 at 10:43 AM Sebastian Benthall < @.***> wrote:

The distribute_params method is designed to let you clone an AgentType while distributing the new Agents over a distribution over a single parameter, such as DiscFac.

https://github.com/econ-ark/HARK/blob/910bff34d6e1d59e686029076ca82a6cf81553c2/HARK/core.py#L1664

Using this feature, I've been surprised when the resulting AgentTypes all have perfectly correlated income shocks.

In hindsight, I should not have been surprised: these AgentTypes all have the same random seed.

However, correcting for this issue required some hacky custom code, looping over the agents, changing their seeds, reseting their RNGs, etc.

While having default and steady random seeds is very useful for testing and development, in Monte Carlo simulations with mutliple runs of the same model, you want actually random random seeds.

HARK could better support this with some kind of flag in the AgentType constructor that, if True, gave the agent a random seed (using the larger computing system to provide the entropy).

A corresponding argument could be added to `distribute_params``

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/econ-ark/HARK/issues/1075, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADKRAFJXERAHHP2ZJQFFRALUIARKXANCNFSM5GOKOAVA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

[sbenthall]

I think the default functionality is very good.

In generally, directly modifying an object's attribute is more messy and error-prone than using a supported API feature.

If you are doing the same thing in all of your applied projects, it should be supported by an API feature.

In a downstream project I was running into an issue where even though I had gone through and set new seeds on the agents and called reset_rng(), for some reason the IncShkDstn random seeds were not updated. I think this had to do with the fact that the IncShkDst was deepcopyd in distribute_params:

https://github.com/sbenthall/HARK_ABM_INTRO_public/issues/31

The fix was cumbersome and given the importance of randomization, and control over it, to various kinds of downstream work it would be better if different workflows were better documented and supported.

https://github.com/sbenthall/HARK_ABM_INTRO_public/commit/29d4ead9d3a9feea2b51f06ed0b9aab5859e445d

[mnwhite]

I think this issue was resolved (and the missing connection to central RNG as well). To the extent there's something left to do, it's on the documentation side, directing users on how to set seeds for AgentType instances.