mnwhite
commented
1 year ago That's the only place that the HARK code itself uses it, but it's meant to be used in research project code. Here's a quick example from a project I have open right now:
[image: image.png]
That's in a project that doesn't use Market, because it's entirely "micro"-- there are no equilibrium objects to solve for.
The multithreading here is ham-handed in that it uses HD writes and reads to send data among temporarily spawned Python processes. It never needs to worry about memory conflicts (etc) because there isn't any shared memory. HD operations are both relatively slow and latent, even with an SSD. It's best used for "big" chunks of work like solving an entire agent type; even then, its best to only "pass back" the necessary information, thus the delSolution() call above-- it throws out the very large solution object and sends back only the simulated data for that type.
On Wed, Mar 22, 2023 at 12:31 PM Sebastian Benthall < @.***> wrote:
We have some support for parallelization using multi-threading:
https://github.com/econ-ark/HARK/blob/876ede3a8de0ba2606a27f6b9b6d2ae42076f580/HARK/parallel.py
The only place this is used is in the Market solve_agents method:
This is a bit brittle; it causes this failure when there's some reasonable refactoring:
1240 (comment)
https://github.com/econ-ark/HARK/pull/1240#issuecomment-1462620763
In general, there are many ways to parallelize and it would be better to have options for how to parallelize, rather than hard-coding one method in the model definition code.
— Reply to this email directly, view it on GitHub https://github.com/econ-ark/HARK/issues/1245, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADKRAFMLOSWAML7FXI7DPRDW5MSNXANCNFSM6AAAAAAWECAZAQ . You are receiving this because you are subscribed to this thread.Message ID: @.***>
We have some support for parallelization using multi-threading:
https://github.com/econ-ark/HARK/blob/876ede3a8de0ba2606a27f6b9b6d2ae42076f580/HARK/parallel.py
The only place this is used is in the Market
solve_agentsmethod: https://github.com/econ-ark/HARK/blob/876ede3a8de0ba2606a27f6b9b6d2ae42076f580/HARK/core.py#L1274-L1301This is a bit brittle; it causes this failure when there's some reasonable refactoring:
https://github.com/econ-ark/HARK/pull/1240#issuecomment-1462620763
In general, there are many ways to parallelize and it would be better to have options for how to parallelize, rather than hard-coding one method in the model definition code.